A taxonomy for reproducible and replicable research in environmental modelling. (December 2020)

• Record Type: Journal Article
• Authors: Essawy, Bakinam T.; Goodall, Jonathan L.; Voce, Daniel; Morsy, Mohamed M.; Sadler, Jeffrey M.; Choi, Young Don; Tarboton, David G.; Malik, Tanu
• Is Part Of: Environmental modelling & software. Volume 134(2020)
• Publication Date: 2020-12
• Subjects: Reproducibility -- Replicability -- Containers -- Docker -- Singularity; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2020.104753
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Despite the growing acknowledgment of reproducibility crisis in computational science, there is still a lack of clarity around what exactly constitutes a reproducible or replicable study in many computational fields, including environmental modelling. To this end, we put forth a taxonomy that defines an environmental modelling study as being either 1) repeatable, 2) runnable, 3) reproducible, or 4) replicable. We introduce these terms with illustrative examples from hydrology using a hydrologic modelling framework along with cyberinfrastructure aimed at fostering reproducibility. Using this taxonomy as a guide, we argue that containerization is an important but lacking component needed to achieve the goal of computational reproducibility in hydrology and environmental modelling. Examples from hydrology are provided to demonstrate how new tools, including a user-friendly tool for containerization of computational analyses called Sciunit, can lower the barrier to reproducibility and replicability in the environmental modelling community. Highlights: Defines a taxonomy describing levels of reproducibility for modelling studies. Levels progress from repeatability to runnability, reproducibility, and replicability. An analysis using the SUMMA hydrologic model is used to demonstrate the taxonomy. Containerization of the conditions of analysis allows for achieving reproducibility. Sciunit software facilitates this containerization of the conditions of analysis.