An ensemble‐based predictive mutation testing approach that considers impact of unreached mutants. (2nd June 2021)

• Record Type: Journal Article
• Authors: Aghamohammadi, Alireza; Mirian‐Hosseinabadi, Seyed‐Hassan
• Is Part Of: Software testing, verification & reliability. Volume 31:Number 7(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-06-02
• Subjects: software testing -- mutation testing -- predictive mutation testing -- machine learning; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1784
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: Predictive mutation testing (PMT) is a technique to predict whether a mutant is killed, using machine learning approaches. Researchers have proposed various methods for PMT over the years. However, the impact of unreached mutants on PMT is not fully addressed. A mutant is unreached if the statement on which the mutant is generated is not executed by any test cases. We aim at showing that unreached mutants can inflate PMT results. Moreover, we propose an alternative approach to PMT, suggesting a different interpretation for PMT. To this end, we replicated the previous PMT research. We empirically evaluated the suggested approach on 654 Java projects provided by prior literature. Our results indicate that the performance of PMT drastically decreases in terms of area under a receiver operating characteristic curve (AUC) from 0.833 to 0.517. Furthermore, PMT performs worse than random guesses on 27% of the projects. The proposed approach improves the PMT results, achieving the average AUC value of 0.613. As a result, we recommend researchers to remove unreached mutants when reporting the results. Abstract : We investigate the impact of unreached mutants on the results of Predictive Mutation Testing (PMT), showing that the performance of PMT decreases significantly. We propose an approach based on Random Forest and Gradient Boosting, considering the effect of unreached mutants and improving the PMT results in terms of various unbiased evaluation metrics. We provide an interpretable explanation for proposed approach, which can be served as guidance for developers to understand why a mutant is killed or alive.