Assessing test suites of extended finite state machines against model‐ and code‐based faults. (18th August 2021)

• Record Type: Journal Article
• Authors: El‐Fakih, K.; Alzaatreh, Ayman; Türker, Uraz Cengiz
• Other Names: Gopinath Rahul guestEditor.; Zhang Jie M. guestEditor.; Kintis Marinos guestEditor.; Papadakis Mike guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 32:Number 7(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-08-18
• Subjects: empirical assessment -- extended finite state machines -- model‐based testing -- mutation scores -- mutation testing; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1789
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: Tests can be derived from extended finite state machine (EFSM) specifications considering the coverage of single‐transfer faults, all transitions using a transition tour, all‐uses, edge‐pair, and prime path with side trip. We provide novel empirical assessments of the effectiveness of these test suites. The first assessment determines for each pair of test suites if there is a difference between the pair in covering EFSM faults of six EFSM specifications. If the difference is found significant, we determine which test suite outperforms the other. The second assessment is similar to the first; yet, it is carried out against code faults of 12 Java implementations of the specifications. Besides, two assessments are provided to determine whether test suites have better coverage of certain classes of EFSM (or code) faults than others. The evaluation uses proper data transformation of mutation scores and p ‐value adjustments for controlling Type I error due to multiple tests. Furthermore, we show that subsuming mutants have an impact on mutation scores of both EFSM and code faults; and accordingly, we use a score that removes them in order not to invalidate the obtained results. The assessments show that all‐uses tests were outperformed by all other tests; transition tours outperformed both edge‐pair and prime path with side trips; and single‐transfer fault tests outperformed all other test suites. Similar results are obtained over the considered EFSM and code fault domains, and there were no significant differences between the test suites coverage of different classes of EFSM and code faults. Abstract : Test suites derived from extended finite state machine specifications are assessed against code‐ and model‐based faults. Results show that single transfer fault tests outperform all other considered test suites; transition tours outperform edge‐pair and prime path with side trips, and all‐uses tests were outperformed by all other test suites. Similar results are obtained over both code‐ and model‐based fault, and there were no significant differences between the test suites coverage of different classes of code‐ and model‐based faults.