Property-based Sensitivity Analysis: An approach to identify model implementation and integration errors. (May 2021)

• Record Type: Journal Article
• Authors: Iwanaga, Takuya; Sun, Xifu; Wang, Qian; Guillaume, Joseph H.A.; Croke, Barry F.W.; Rahman, Joel; Jakeman, Anthony J.
• Is Part Of: Environmental modelling & software. Volume 139(2021)
• Publication Date: 2021-05
• Subjects: Integrated development cycle -- Diagnostic testing -- Sensitivity analysis -- Parameter sensitivity -- Parameter inactivity -- Integrated environmental model; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2021.105013
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Diagnostic testing is an oft-recommended use of sensitivity analysis to assess correctness or plausibility of model behavior. In this paper we demonstrate the use of sensitivity analysis as a complementary first-pass software test for the validation of model behavior. Typical testing processes rely on comparing model outputs to results known to be correct. Such approaches are limited to specific model configurations and require that correct results be known in advance. Property-based Sensitivity Analysis (PbSA) examines model properties in terms of the behavior of parameter sensitivities to provide a line of evidence that the expected conceptual relationships between model factors and their effects are present. Unanticipated results can indicate issues to be corrected. The PbSA approach is also scalable as it can complement existing testing practices and be applied in conjunction with global sensitivity methods that can reuse existing model evaluations or are otherwise independent of the sampling scheme. Highlights: Property-based testing is a common approach for diagnostic assessment and validation in software development. Sensitivity analysis is used for similar purposes in model development. Realities of integrated model development may complicate the use of global sensitivity analyses for diagnostic purposes. Parameter sensitivities can be treated as a property to test to aid in identifying issues during the model development cycle. Sensitivity analyses should be leveraged to complement other model testing and validation processes.