An open software package for data reconciliation and gap filling in preparation of Water and Resource Recovery Facility Modeling. (September 2018)

• Record Type: Journal Article
• Authors: De Mulder, C.; Flameling, T.; Weijers, S.; Amerlinck, Y.; Nopens, I.
• Is Part Of: Environmental modelling & software. Volume 107(2018)
• Page Start: 186
• Page End: 198
• Publication Date: 2018-09
• Subjects: Data analysis -- Data imputation -- Online data; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2018.05.015
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: High quality data is of crucial importance for model development: it provides a model input and is a prerequisite for model calibration and validation. Data reconciliation is often a very time-consuming task, so even when on-line data is available, the option is often chosen to synthetically generate data, losing a lot of information contained in the available data. This contribution showcases a Python™ package that allows a streamlined work-flow and provides possibilities for data analysis, validation and gap filling, with as main goals to use as much of the data as possible and to fill gaps in the data with a known reliability. This provides a means towards more data use and a more sound calibration and validation, while significantly reducing time spent on data reconciliation. The package is published and made openly available on GitHub. This avoids multiple implementations while being accessible to the community for suggested improvements. Highlights: Data analysis is time-consuming, leading to the use of generated data as model input and thus to loss of information. A Python software package with a flexible workflow is presented to analyze on-line data and fill gaps caused by filtering. The filling algorithms implemented can be checked for their reliability when used to fill gaps in the data at hand. The main impact for the user is increased use of information contained in the data and time saving. The package is made openly available on GitHub and PyPI under an open GNU-GPLv3.0 license.