Mutation Analysis for System of Systems Policy Testing

Abstract

A System of Systems (SoS) is a set of the constituent systems (CS) which has managerial and operational independence. To address an SoS-level goal that cannot be satisfied by each CS, an SoS policy guides or forces the CSs to collaborate with each other. If there is a fault in the SoS policy, SoS may fail to reach its goal, even if there is no fault in the CSs. Such a call for SoS policy testing leads to an essential question - how can testers evaluate the effectiveness of test cases? In this paper, we suggest a mutation analysis approach for SoS policy testing. Mutation analysis is a systematic way of evaluating test cases using artificial faults called mutants. As a general mutation framework for SoS policy testing, we present an overview of mutation analysis in SoS policy testing as well as the key aspects that must be defined in practice. To demonstrate the applicability of the proposed approach, we provide a case study using a traffic management SoS with the Simulation of Urban Mobility (SUMO) simulator. The results show that the mutation analysis is effective at evaluating fault detection effectiveness of test cases for SoS policies at a reasonable cost.