A sensitivity analysis of the PAWN sensitivity index. (May 2020) - Puy, Arnald; Lo Piano, Samuele; Saltelli, Andrea
Abstract: The PAWN index is gaining traction among the modelling community as a sensitivity measure. However, the robustness to its design parameters has not yet been scrutinized: the size ( N ) and sampling ( ε ) of the model output, the number of conditioning intervals ( n ) or the summary statistic ( θ ). Here we fill this gap by running a sensitivity analysis of a PAWN-based sensitivity analysis. We compare the results with the design uncertainties of the Sobol' total-order index ( S T i ∗ ). Unlike in S T i ∗, the design uncertainties in PAWN create non-negligible chances of producing biased results when ranking or screening inputs. The dependence of PAWN upon ( N, n, ε, θ ) is difficult to tame, as these parameters interact with one another. Even in an ideal setting in which the optimum choice for ( N, n, ε, θ ) is known in advance, PAWN might not allow to distinguish an influential, non-additive model input from a truly non-influential model input. Highlights: We conduct a sensitivity analysis of the PAWN sensitivity index. We compare the results with the Sobol' total-order index. PAWN is more sensitive to its design parameters. This sensitivity increases the risk of PAWN yielding biased results.