Measuring and specifying combinatorial coverage of test input configurations. (December 2016)

• Record Type: Journal Article
• Authors: Kuhn, D.; Kacker, Raghu; Lei, Yu
• Is Part Of: Innovations in systems and software engineering. Volume 12:Number 4(2016)
• Page Start: 249
• Page End: 261
• Publication Date: 2016-12
• Subjects: Combinatorial testing -- Configuration model -- Covering array -- Fault coverage -- State-space coverage -- T-way testing -- Verification and validation (V&V); Software engineering -- Periodicals; Systems engineering -- Periodicals; Génie logiciel -- Périodiques; Ingénierie des systèmes -- Périodiques; Electronic journals; 005.1
• DOI: 10.1007/s11334-015-0266-2
• Languages: English
• ISSNs: 1614-5046

Abstract

Abstract A key issue in testing ishow many tests are needed for a required level of coverage or fault detection . Estimates are often based on error rates in initial testing, or on code coverage. For example, tests may be run until a desired level of statement or branch coverage is achieved. Combinatorial methods present an opportunity for a different approach to estimating required test set size, using characteristics of the test set. This paper describes methods for estimating the coverage of, and ability to detect, t -way interaction faults of a test set based on a covering array. We also develop a connection between (static) combinatorial coverage and (dynamic) code coverage, such that if a specific condition is satisfied, 100 % branch coverage is assured. Using these results, we propose practical recommendations for using combinatorial coverage in specifying test requirements, and for improving estimates of the fault detection capacity of a test set.