Case Study Investigation of the Fault Detection and Error Locating Effects of Architecture-based Software Testing

Abstract

For software testing, it is well known that the architecture of a software system can be utilized to enhance testability, fault detection and error locating. However, how much and what effects architecture-based software testing has on software testing have been rarely studied. Thus, this paper undertakes case study investigation of the effects of architecture-based software testing specifically with respect to fault detection and error locating. Through comparing the outcomes with the conventional testing approaches that are not based on test architectures, we confirm the effectiveness of architecture-based software testing with respect to fault detection and error locating. The case studies show that using test architecture can improve fault detection rate by 44.1%-88.5% and reduce error locating time by 3%-65.2%, compared to the conventional testing that does not rely on test architecture. With regard to error locating, the scope of relevant components or statements was narrowed by leveraging test architecture for approximately 77% of the detected faults. We also show that architecture-based testing could provide a means of defining an exact oracle or oracles with range values. This study shows by way of case studies the extent to which architecture-based software testing can facilitate detecting certain types of faults and locating the errors that cause such faults. In addition, we discuss the contributing factors of architecture-based software testing which enable such enhancement in fault detection and error locating.