A framework for characterising and evaluating the effectiveness of environmental modelling. (August 2019) - Hamilton, Serena H.; Fu, Baihua; Guillaume, Joseph H.A.; Badham, Jennifer; Elsawah, Sondoss; Gober, Patricia; Hunt, Randall J.; Iwanaga, Takuya; Jakeman, Anthony J.; Ames, Daniel P.; Curtis, Allan; Hill, Mary C.; et al.
Abstract: Environmental modelling is transitioning from the traditional paradigm that focuses on the model and its quantitative performance to a more holistic paradigm that recognises successful model-based outcomes are closely tied to undertaking modelling as a social process, not just as a technical procedure. This paper redefines evaluation as a multi-dimensional and multi-perspective concept, and proposes a more complete framework for identifying and measuring the effectiveness of modelling that serves the new paradigm. Under this framework, evaluation considers a broader set of success criteria, and emphasises the importance of contextual factors in determining the relevance and outcome of the criteria. These evaluation criteria are grouped into eight categories: project efficiency, model accessibility, credibility, saliency, legitimacy, satisfaction, application, and impact. Evaluation should be part of an iterative and adaptive process that attempts to improve model-based outcomes and foster pathways to better futures. Highlights: An evaluation framework for the new process-oriented paradigm of modelling is presented. Effectiveness of modelling is a multi-dimensional and multi-perspective concept. 32 criteria for model evaluation are considered from project-level to system-level outcomes. We link the success of modelling to modelling context and modelling practices.