How to keep it adequate: A protocol for ensuring validity in agent-based simulation. (January 2023)

• Record Type: Journal Article
• Authors: Troost, Christian; Huber, Robert; Bell, Andrew R.; van Delden, Hedwig; Filatova, Tatiana; Le, Quang Bao; Lippe, Melvin; Niamir, Leila; Polhill, J. Gareth; Sun, Zhanli; Berger, Thomas
• Is Part Of: Environmental modelling & software. Volume 159(2023)
• Publication Date: 2023-01
• Subjects: Model validation -- Model inference -- Calibration -- Generalisation -- Regime shift; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2022.105559
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: There has so far been no shared understanding of validity in agent-based simulation. We here conceptualise validation as systematically substantiating the premises on which conclusions from simulation analysis for a particular modelling context are built. Given such a systematic perspective, validity of agent-based models cannot be ensured if validation is merely understood as an isolated step in the modelling process. Rather, valid conclusions from simulation analysis require context-adequate method choices at all steps of the simulation analysis including model construction, model and parameter inference, uncertainty analysis and simulation. We present a twelve-step protocol to highlight the (often hidden) premises for methodological choices and their link to the modelling context. It is designed to aid modelers in understanding their context and in choosing and documenting context-adequate and mutually consistent methods throughout the modelling process. Its purpose is to assist reviewers and the community as a whole in assessing and discussing context-adequacy. Graphical abstract: Image 1 Highlights: A comprehensive understanding of validation for agent-based models and beyond. 11 dimensions to characterise the modelling context and purpose. A characterisation of the premises of common validation approaches. A detailed protocol to guide context-adequate model construction and review. A consistent tracking of uncertainty propagation through the modelling process.