A new method for analysing the interrelationship between performance indicators with an application to agrometeorological models. (November 2015)

• Record Type: Journal Article
• Authors: Sanna, Mattia; Bellocchi, Gianni; Fumagalli, Mattia; Acutis, Marco
• Is Part Of: Environmental modelling & software. Volume 73(2015:Nov.)
• Page Start: 286
• Page End: 304
• Publication Date: 2015-11
• Subjects: Model evaluation -- Performance indicators -- Stable correlation; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2015.08.017
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: The use of a variety of metrics is advocated to assess model performance but correlated metrics may convey the same information, thus leading to redundancy. Starting from this assumption, a method was developed for selecting, from among a collection of performance indicators, one or more subsets providing the same information as the entire set. The method, based on the definition of "stable correlation", was applied to 23 performance indicators of agrometeorological models, calculated on large sets of simulated and observed data of four agronomic and meteorological variables: above-ground biomass, leaf area index, hourly air relative humidity and daily solar radiation. Two subsets were determined: {Squared Bias, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index, Modified Modelling Efficiency}, {Persistence Model Efficiency, Root Mean Squared Relative Error, Coefficient of Determination, Pattern Index}. The method needs corroboration but is statistically founded and can support the implementation of standardized evaluation tools. Highlights: The correlation between modelling evaluation metrics was analysed on 4 datasets. The concept of stable correlation among indicators was developed. A methodology to select metrics is proposed, based on stable correlation definition. {SB, RMSRE, CD, PI, EF1} and {PME, RMSRE, CD, PI} are the subsets optimized for our data sets.