On strong mutation and the theory of subsuming logic‐based mutants. (19th April 2018)

• Record Type: Journal Article
• Authors: Lindström, Birgitta; Márki, András
• Other Names: Just René guestEditor.; Krinke Jens guestEditor.; Li Nan guestEditor.; Rojas José Miguel guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 29:Number 1/2(2019)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2018-04-19
• Subjects: error propagation -- mutant subsumption -- mutation testing -- redundant mutants; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1667
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: Redundant mutants might cause problems when benchmarking since testing techniques can get high scores without detecting any nonredundant mutants. However, removing nonredundant mutants might cause similar problems. Subsumed mutants are per definition also redundant since no additional tests are required to detect them once all other mutants are detected. We focus on relational operator replacement (ROR) and conditional operator replacement mutants. Subsumption relations between ROR mutants are defined by fault hierarchies. The fault hierarchies are proven for weak mutation but have since they were published been used with strong mutation. We prove that ROR fault hierarchies do not hold for strong mutation and show why. We also show that the probability for a random test to experience the problem can be more than 30% and that 50% of the mutants might be affected in a real software system. Finally, we show that there is a similar problem with the theory on sufficient conditional operator replacement. Abstract : Reduced ROR avoids redundant mutants while generating all nonredundant mutants, using defined subsumption relations. Our experiments show that the probability for a test to detect a mutant m and still fail to detect a mutant, which should be subsumed by m, can be above 30% when reduced ROR is used together with strong mutation. We present a root cause and show that this root cause applied to more than 50% of the ROR mutants in a study with real‐sized software.