A sensitivity analysis of the PAWN sensitivity index. (May 2020)

• Record Type: Journal Article
• Authors: Puy, Arnald; Lo Piano, Samuele; Saltelli, Andrea
• Is Part Of: Environmental modelling & software. Volume 127(2020)
• Publication Date: 2020-05
• Subjects: Uncertainty -- Environmental modelling -- Statistics -- Risk; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2020.104679
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: The PAWN index is gaining traction among the modelling community as a sensitivity measure. However, the robustness to its design parameters has not yet been scrutinized: the size ( N ) and sampling ( ε ) of the model output, the number of conditioning intervals ( n ) or the summary statistic ( θ ). Here we fill this gap by running a sensitivity analysis of a PAWN-based sensitivity analysis. We compare the results with the design uncertainties of the Sobol' total-order index ( S T i ∗ ). Unlike in S T i ∗, the design uncertainties in PAWN create non-negligible chances of producing biased results when ranking or screening inputs. The dependence of PAWN upon ( N, n, ε, θ ) is difficult to tame, as these parameters interact with one another. Even in an ideal setting in which the optimum choice for ( N, n, ε, θ ) is known in advance, PAWN might not allow to distinguish an influential, non-additive model input from a truly non-influential model input. Highlights: We conduct a sensitivity analysis of the PAWN sensitivity index. We compare the results with the Sobol' total-order index. PAWN is more sensitive to its design parameters. This sensitivity increases the risk of PAWN yielding biased results.