Improved validation framework and R-package for artificial neural network models. (June 2017)

• Record Type: Journal Article
• Authors: Humphrey, Greer B.; Maier, Holger R.; Wu, Wenyan; Mount, Nick J.; Dandy, Graeme C.; Abrahart, Robert J.; Dawson, Christian W.
• Is Part Of: Environmental modelling & software. Volume 92(2017)
• Page Start: 82
• Page End: 106
• Publication Date: 2017-06
• Subjects: Artificial neural networks -- Multi-layer perceptron -- R-package -- Structural validation -- Replicative validation -- Predictive validation; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2017.01.023
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Validation is a critical component of any modelling process. In artificial neural network (ANN) modelling, validation generally consists of the assessment of model predictive performance on an independent validation set (predictive validity). However, this ignores other aspects of model validation considered to be good practice in other areas of environmental modelling, such as residual analysis (replicative validity) and checking the plausibility of the model in relation to a priori system understanding (structural validity). In order to address this shortcoming, a validation framework for ANNs is introduced in this paper that covers all of the above aspects of validation. In addition, thevalidann R-package is introduced that enables these validation methods to be implemented in a user-friendly and consistent fashion. The benefits of the framework and R-package are demonstrated for two environmental modelling case studies, highlighting the importance of considering replicative and structural validity in addition to predictive validity. Highlights: A comprehensive validation framework for ANNs is proposed. The 'validann' R-package for implementing the validation framework is introduced. Application of the framework and R-package is demonstrated on two real case studies. Results reveal that predictively valid ANN models may not be credible. Adoption of the framework leads to improvements in overall ANN validity.