A non-parametric bootstrapping framework embedded in a toolkit for assessing water quality model performance. (September 2018)

• Record Type: Journal Article
• Authors: Libera, Dominic A.; Sankarasubramanian, A.; Sharma, Ashish; Reich, Brian J.
• Is Part Of: Environmental modelling & software. Volume 107(2018)
• Page Start: 25
• Page End: 33
• Publication Date: 2018-09
• Subjects: Water quality modeling -- Performance assessment -- Non-parametric re-sampling; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2018.05.013
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Assessing the ability to predict nutrient concentration in streams is important for determining compliance with the Numeric Nutrient Water Quality Criteria for Nitrogen in the U.S.A. Evaluation of the USGS's Load Estimator (LOADEST) and the Weighted Regression on Time, Discharge, and Season (WRTDS) models in predicting total nitrogen loads over 18 stations from the Water Quality Network show good performance (Nash-Sutcliffe Efficiency (NSE) > 0.8) in capturing the observed variability even for stations with limited data. However, both models captured only 40% of observed variance in total nitrogen (TN) concentration (NSE < 0.4). Thus, the same dataset performed differently in predicting two attributes – TN load and concentration – questioning the predictive skill of the models. This study proposes a non-parametric re-sampling approach for assessing the performance of water quality models particularly in predicting TN concentration. Null distributions for three common performance metrics belonging to populations of metrics with no skill in capturing the observed variability are constructed through a bootstrap resampling technique. Sample metrics from the LOADEST and WRTDS model in predicting TN concentration are used to calculate p-values for determining if the sample metrics belongs to the null distributions. . Highlights: A non-parametric framework is presented for building null distributions of three performance metrics having no skill. A different skill score is proposed for providing more numerical comparison between performance metrics and type of model. Concentration estimates from the LOADEST and WRTDS models are statistically different than zero. Software is available for free as a windows application or MATLAB toolkit that communicates with FORTRAN and R.