Covering and Uncovering Equivalent Mutants. (4th April 2012)

• Record Type: Journal Article
• Authors: Schuler, David; Zeller, Andreas
• Other Names: Cavalli Ana guestEditor.; Ghosh Sudipto guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 23:Number 5(2013)
• Page Start: 353
• Page End: 374
• Publication Date: 2012-04-04
• Subjects: mutation testing -- code coverage -- dynamic analysis; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1473
• Languages: English
• ISSNs: 0960-0833

Abstract

SUMMARY: Mutation testing measures the adequacy of a test suite by seeding artificial defects (mutations) into a program. If a test suite fails to detect a mutation, it may also fail to detect real defects—and hence should be improved. However, there are also mutations that keep the program semantics unchanged and thus cannot be detected by any test suite. Such equivalent mutants must be weeded out manually, which is a tedious task. In this paper, we examine whether changes in coverage can be used to detect non‐equivalent mutants: If a mutant changes the coverage of a run, it is more likely to be non‐equivalent. In a sample of 140 manually classified mutations of seven Java programs with 5000 to 100 000 lines of code, we found that (i) the problem is serious and widespread—about 45% of all undetected mutants turned out to be equivalent; (ii) manual classification takes time—about 15 min per mutation; (iii) coverage is a simple, efficient and effective means to identify equivalent mutants—with a classification precision of 75% and a recall of 56%; and (iv) coverage as an equivalence detector is superior to the state of the art, in particular violations of dynamic invariants. Our detectors have been released as part of the open‐source JAVALANCHE framework; the data set is publicly available for replication and extension of experiments. Copyright © 2012 John Wiley & Sons, Ltd. Abstract : In this paper, we examine whether changes in coverage can be used to detect non‐equivalent mutants. In a sample of 140 manually classified mutations of seven Java programs with 5000 to 100 000 lines of code, we found that: the problem is serious and widespread (45% of all undetected mutants are equivalent); manual classification takes time (15 min per mutation); coverage is a simple and effective means to identify equivalent mutants (with a classification precision of 75% and 56% recall).