Checked coverage: an indicator for oracle quality. (8th May 2013)

• Record Type: Journal Article
• Authors: Schuler, David; Zeller, Andreas
• Other Names: Harman Mark guestEditor.; Korel Bogdan guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 23:Number 7(2013)
• Page Start: 531
• Page End: 551
• Publication Date: 2013-05-08
• Subjects: test suite quality -- coverage metrics -- dynamic slicing -- mutation testing; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1497
• Languages: English
• ISSNs: 0960-0833

Abstract

SUMMARY: A known problem of traditional coverage metrics is that they do not assess oracle quality —that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept of checked coverage —the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open‐source projects show that checked coverage is a sure indicator for oracle quality and even more sensitive than mutation testing. Copyright © 2013 John Wiley & Sons, Ltd. Abstract : A known problem of traditional coverage metrics is that they do not assess oracle quality —that is, whether the computation result is actually checked against expectations. In this paper, we introduce the concept of checked coverage, the dynamic slice of covered statements that actually influence an oracle. Our experiments on seven open‐source projects show that checked coverage is a sure indicator for oracle quality and even more sensitive than mutation testing. Copyright ©2013 John Wiley & Sons, Ltd.