Cause reduction: delta debugging, even without bugs. (31st March 2015)

• Record Type: Journal Article
• Authors: Groce, Alex; Alipour, Mohammad Amin; Zhang, Chaoqiang; Chen, Yang; Regehr, John
• Is Part Of: Software testing, verification & reliability. Volume 26:Number 1(2016)
• Page Start: 40
• Page End: 68
• Publication Date: 2015-03-31
• Subjects: test case minimization -- regression testing -- delta debugging -- code coverage -- software testing; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1574
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: What is a test case for ? Sometimes, to expose a fault. Tests can also exercise code, use memory or time, or produce desired output. Given a desired effect, a test case can be seen as a cause, and its components divided into essential (required for effect) and accidental. Delta debugging is used for removing accidents from failing test cases, producing smaller test cases that are easier to understand. This paper extends delta debugging by simplifying test cases with respect to arbitrary effects, a generalization called cause reduction . Suites produced by cause reduction provide effective quick tests for real‐world programs. For Mozilla's JavaScript engine, the reduced suite is possibly more effective for finding faults. The effectiveness of reduction‐based suites persists through changes to the software, improving coverage by over 500 branches for versions up to 4 months later. Cause reduction has other applications, including improving seeded symbolic execution, where using reduced tests can often double the number of additional branches explored. Copyright © 2015 John Wiley & Sons, Ltd. Abstract : This paper presents a generalization of delta debugging (cause reduction) in which test cases are reduced based on an arbitrary property, rather than only with respect to ability to trigger a fault. The graph shows that basing test suites on tests cases that have been reduced with respect to statement coverage can result in much better code coverage for very small regression tests than using unreduced test cases. Cause reduction can also benefit symbolic execution and stress testing.