Why so many published sensitivity analyses are false: A systematic review of sensitivity analysis practices. (April 2019)

• Record Type: Journal Article
• Authors: Saltelli, Andrea; Aleksankina, Ksenia; Becker, William; Fennell, Pamela; Ferretti, Federico; Holst, Niels; Li, Sushan; Wu, Qiongli
• Is Part Of: Environmental modelling & software. Volume 114(2019)
• Page Start: 29
• Page End: 39
• Publication Date: 2019-04
• Subjects: Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2019.01.012
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Sensitivity analysis provides information on the relative importance of model input parameters and assumptions. It is distinct from uncertainty analysis, which addresses the question 'How uncertain is the prediction?' Uncertainty analysis needs to map what a model does when selected input assumptions and parameters are left free to vary over their range of existence, and this is equally true of a sensitivity analysis. Despite this, many uncertainty and sensitivity analyses still explore the input space moving along one-dimensional corridors leaving space of the input factors mostly unexplored. Our extensive systematic literature review shows that many highly cited papers (42% in the present analysis) fail the elementary requirement to properly explore the space of the input factors. The results, while discipline-dependent, point to a worrying lack of standards and recognized good practices. We end by exploring possible reasons for this problem, and suggest some guidelines for proper use of the methods. Highlights: A systematic review of 280 scientific papers mentioning sensitivity analysis has been performed. The analysis addresses the use of SA in the context of mathematical modelling, focusing on highly cited works. Many highly-cited papers (42% in the present analysis) present a SA of poor quality. The results, while discipline-dependent, point to a worrying lack of standards and good practices. Some guidelines for proper use of the methods are suggested.