Estimating model prediction error: Should you treat predictions as fixed or random?. (October 2016)

• Record Type: Journal Article
• Authors: Wallach, Daniel; Thorburn, Peter; Asseng, Senthold; Challinor, Andrew J.; Ewert, Frank; Jones, James W.; Rotter, Reimund; Ruane, Alex
• Is Part Of: Environmental modelling & software. Volume 84(2016:Oct.)
• Page Start: 529
• Page End: 539
• Publication Date: 2016-10
• Subjects: Crop model -- Uncertainty -- Prediction error -- Parameter uncertainty -- Input uncertainty -- Model structure uncertainty; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2016.07.010
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Crop models are important tools for impact assessment of climate change, as well as for exploring management options under current climate. It is essential to evaluate the uncertainty associated with predictions of these models. We compare two criteria of prediction error; MSEP fixed, which evaluates mean squared error of prediction for a model with fixed structure, parameters and inputs, and MSEP uncertain (X), which evaluates mean squared error averaged over the distributions of model structure, inputs and parameters. Comparison of model outputs with data can be used to estimate the former. The latter has a squared bias term, which can be estimated using hindcasts, and a model variance term, which can be estimated from a simulation experiment. The separate contributions to MSEP uncertain (X) can be estimated using a random effects ANOVA. It is argued that MSEP uncertain (X) is the more informative uncertainty criterion, because it is specific to each prediction situation. Highlights: It is important to estimate the uncertainty in crop model predictions. Two uncertainty criteria are defined, treating predictions as fixed or random. The random criterion includes model, parameter and input uncertainty and also bias. The random prediction criterion is specific for each prediction situation.