Comparing model-based and dynamic event-extraction based GUI testing techniques: An empirical study

Abstract

Graphical user interfaces are pervasive in modern software systems, and to ensure their quality it is important to test them. Two primary classes of automated GUI testing approaches, those based on static models and those based on dynamic event-extraction, present tradeoffs in cost and effectiveness. For example, static model-based GUI testing techniques can create test cases that contain nonexecutable events, whereas dynamic event-extraction based GUI testing techniques can create larger numbers of duplicate test cases. To better understand the effects of these tradeoffs, we created a GUI testing framework that facilitates fair comparison of different GUI testing techniques, and we conducted a controlled experiment comparing representative versions of static-model based and dynamic event-extraction based testing techniques on several GUI-based Java applications. Our study reveals several cost and effectiveness tradeoffs between the techniques, with implications for research and practice.