Partition-based coverage metrics and type-guided search in concolic testing for JavaScript applications

Abstract

JavaScript extends its uses from client-side web applications to mobile applications, but testing JavaScript applications is not yet satisfactory. Researchers have not spent much attention on testing JavaScript programs, and existing coverage metrics and testing mechanisms for C and Java may not be applicable to JavaScript because of its extremely dynamic semantics without any compile-time checks. Because, in JavaScript, any variable may have any kinds of types during evaluation, and because varying types change program execution flows, testing JavaScript programs requires more cases to cover different execution flows. Thus, test cases with 100% coverage levels in terms of the existing coverage metrics for statically typed languages may miss faults in JavaScript. Moreover, existing search strategies for statically typed languages may not exercise increased test requirements by dynamic languages effectively. In this paper, we identify JavaScript characteristics that make thorough testing of JavaScript programs more difficult than testing C and Java programs. To address such characteristics, we propose new partition-based coverage metrics that expose implicit execution flows using varying types. To generate test cases satisfying the coverage metrics effectively, we develop type-guided search strategies for concolic testing using program analysis results. We evaluate the new coverage metrics and search strategies for concolic testing with open-source web applications, and the preliminary experimental results show that their practical uses in JavaScript testing are promising.