Unpacking the modelling process via sensitivity auditing. (December 2022)

• Record Type: Journal Article
• Authors: Lo Piano, Samuele; Sheikholeslami, Razi; Puy, Arnald; Saltelli, Andrea
• Is Part Of: Futures. Volume 144(2022)
• Publication Date: 2022-12
• Subjects: Model quality assessment -- Science-policy interface -- Ethics of quantification -- Sensitivity analysis -- Uncertainty analysis -- Transparency; Economic forecasting -- Periodicals; Technological forecasting -- Periodicals; Economic policy -- Periodicals; Prévision économique -- Périodiques; Prévision technologique -- Périodiques; Politique économique -- Périodiques; Economic forecasting; Economic policy; Technological forecasting; Periodicals; Electronic journals; 330.0112
• DOI: 10.1016/j.futures.2022.103041
• Languages: English
• ISSNs: 0016-3287

Abstract

Abstract: Acknowledging the conditionality of model-based evidence facilitates the dialogue between model developers and model users, especially when models are used to guide decisions at the science-policy interface. In general, model users have limited access to verify the realism of a model, being only exposed to model plausibility and trustworthiness; instead, modellers have an an array of validation and verification techniques available. In the end, model credibility is what both developers and users aim for, also in the interest of shielding from the possible pitfall of over-interpreting the model results. To this end, in this contribution we discuss sensitivity auditing, an extension of sensitivity analysis, that can help model developers and users to overcome communication barriers and foster dialogue around modelling activities. The use of sensitivity auditing is not limited to models in a restricted sense, but it can be applied to any policy-relevant instance of quantification, including metrics, rankings and indicators. We present six real-world applications of sensitivity auditing to instances of quantification in a range of socio-environmental systems, including public health, education, and the water-food nexus. These examples reveal the usefulness of sensitivity auditing in facilitating the proper use of numbers and models at the science-policy-society interface and in avoiding uncertainty laundering. Highlights: Sensitivity auditing is proposed to scrutinise socio-environmental quantifications. The features of the sensitivity auditing checklist are discussed against other methods. The potential of the approach is showcased in six socio-environmental examples. Room for improvement in modelling activities is identified along with ways forward. Sensitivity auditing is a promising tool within a framework for ethics of quantification.