Higher accuracy and lower run time: efficient mutation analysis using non‐redundant mutation operators. (11th December 2014)

• Record Type: Journal Article
• Authors: Just, René; Schweiggert, Franz
• Other Names: Jia Yue guestEditor.; Merayo Mercedes guestEditor.; Harman Mark guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 25:Number 5/7(2015)
• Page Start: 490
• Page End: 507
• Publication Date: 2014-12-11
• Subjects: mutation analysis -- redundant mutants -- test adequacy -- code coverage; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1561
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: Mutation analysis is a powerful but computationally expensive method to measure the effectiveness of a testing or debugging technique. The high cost is due, in part, to redundant mutants generated by commonly used mutation operators. A mutant is said to be redundant if its outcome can be predicted based on the outcome of other mutants. The execution of those redundant mutants is unnecessary and wastes CPU resources. Moreover, the inclusion of redundant mutants may lead to a skewed mutant detection rate and therefore misrepresent the effectiveness of the assessed testing or debugging technique. This paper extends previous work and makes the following contributions. First, it defines and provides non‐redundant versions of the conditional operator replacement, unary operator insertion, and relational operator replacement mutation operators. Second, it reports on a conducted empirical study using 10 real‐world programmes that comprise a total of 410 000 lines of code. The empirical study used developer‐written and generated test suites. The results show how prevalent redundant mutants are and how their elimination improves the efficiency and accuracy of mutation analysis. In summary, the total mutation analysis run time decreased by more than 20% by removing redundant mutants, and the inclusion of redundant mutants led to an overestimated mutation score for all analysed test suites. Copyright © 2014 John Wiley & Sons, Ltd. Abstract : This paper studies redundancies in commonly used mutation operators and provides non‐redundant versions of the conditional operator replacement, unary operator insertion, and relational operator replacement mutation operators. The empirical study in this paper shows how the inclusion redundant mutants affect the efficiency and accuracy of mutation analysis. In summary, the total mutation analysis run time decreased by more than 20% by removing redundant mutants, and the inclusion of redundant mutants led to an overestimated mutation score for all analyzed test suites.