Causal inference based fault localization for numerical software with NUMFL. (28th November 2016)

• Record Type: Journal Article
• Authors: Bai, Zhuofu; Shu, Gang; Podgurski, Andy
• Other Names: Fraser Gordon guestEditor.; Marinov Darko guestEditor.
• Is Part Of: Software testing, verification & reliability. Volume 27:Number 6(2017)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2016-11-28
• Subjects: value‐based statistical fault localization -- causal inference -- confounding bias -- generalized propensity score -- covariate‐balancing propensity score -- failure‐causing effect estimation; Computer software -- Testing -- Periodicals; Computer software -- Verification -- Periodicals; Computer software -- Reliability -- Periodicals; 005.14
• DOI: 10.1002/stvr.1613
• Languages: English
• ISSNs: 0960-0833

Abstract

Summary: This paper presents NUMFL, a value‐based causal inference technique for localizing faults in numerical software. NUMFL combines causal and statistical analyses to characterize the causal effects of individual numerical expressions on output errors. Given value‐profiles for an expression's variables, NUMFL uses generalized propensity scores or covariate balancing propensity scores to reduce confounding bias caused by evaluation of other, faulty expressions. It estimates the average failure‐causing effect of an expression using statistical regression models fit within generalized propensity score or covariate balancing propensity score subclasses (strata). This paper also reports on an empirical evaluation of NUMFL involving components from four Java numerical libraries, in which it was compared with five alternative statistical fault localization metrics. The results indicate that NUMFL is more effective than competitive statistical fault localization techniques. The results also indicate NUMFL that works surprisingly well with data from failing runs alone. Copyright © 2016 John Wiley & Sons, Ltd. Abstract : This paper presents NUMFL, a value‐based causal inference technique for localizing faults in numerical software. Given value profiles for an expression's variables, NUMFL uses generalized propensity scores (GPSs) or covariate balancing propensity scores (CBPSs) to reduce confounding bias caused by evaluation of other faulty expressions. It estimates the average failure‐causing effect of an expression using statistical regression models. The empirical results indicate that NUMFL is more effective than competitive metrics, and NUMFL works surprisingly well with data from failing runs alone.