(A) data flow-based structural testing technique for FBD programs

Abstract

With increased use of programmable logic controllers (PLCs) in implementing safety critical systems, quality assurance of PLC programs became an important issue. Although regulation authorities require structural testing be performed for safety critical system software by identifying coverage criteria to be satisfied and accomplishment measured, there has been little research on structural testing or test coverage criteria applicable to Function Block Diagram (FBD), a PLC programming language widely used in industry. Classical test coverage criteria, based on control flow graphs, are inadequate when applied to a data flow language FBD. This thesis proposes new structural test coverage criteria focusing on data flow-centric characteristics of FBD programs. Interpreting an FBD program as a data flow graph, we define basic coverage, input condition coverage, and complex condition coverage criteria after defining data path and data path condition. We present $\it{FBDTestMeasurer}$, an automatic test coverage measurement case tool for FBD programs. Given a set of test cases and the selected test coverage criteria, $\it{FBDTestMeasurer}$ provides automatically measured coverage result and uncovered test requirements. Analysis of uncovered test requirements can assist in identifying inadequately tested parts and generating additional test cases. We also present automatic test case generation procedure for FBD programs. Test cases satisfying test requirements represented by logical formulas are generated utilizing a Satisfiability Modulo Theories (SMT) solver, $\it{Yices}$, in our approach. Case study conducted using a real-world reactor protection system demonstrated the effectiveness of the proposed approach. Using test cases that had been manually prepared by FBD testing professionals, our automatic coverage measurement technique found many aspects of the FBD logic that were not tested sufficiently. Automatic test case generation applied to i...