A cross-validation package driving Netica with python. (January 2015)

• Record Type: Journal Article
• Authors: Fienen, Michael N.; Plant, Nathaniel G.
• Is Part Of: Environmental modelling & software. Volume 63(2015:Jan.)
• Page Start: 14
• Page End: 23
• Publication Date: 2015-01
• Subjects: Cross-validation -- Bayesian networks -- Uncertainty -- Probability -- Python -- Netica -- Prediction; Environmental monitoring -- Computer programs -- Periodicals; Ecology -- Computer simulation -- Periodicals; Digital computer simulation -- Periodicals; Computer software -- Periodicals; Environmental Monitoring -- Periodicals; Computer Simulation -- Periodicals; Environnement -- Surveillance -- Logiciels -- Périodiques; Écologie -- Simulation, Méthodes de -- Périodiques; Simulation par ordinateur -- Périodiques; Logiciels -- Périodiques; Computer software; Digital computer simulation; Ecology -- Computer simulation; Environmental monitoring -- Computer programs; Periodicals; Electronic journals; 363.70015118
• DOI: 10.1016/j.envsoft.2014.09.007
• Languages: English
• ISSNs: 1364-8152

Abstract

Abstract: Bayesian networks (BNs) are powerful tools for probabilistically simulating natural systems and emulating process models. Cross validation is a technique to avoid overfitting resulting from overly complex BNs. Overfitting reduces predictive skill. Cross-validation for BNs is known but rarely implemented due partly to a lack of software tools designed to work with available BN packages. CVNetica is open-source, written in Python, and extends the Netica software package to perform cross-validation and read, rebuild, and learn BNs from data. Insights gained from cross-validation and implications on prediction versus description are illustrated with: a data-driven oceanographic application; and a model-emulation application. These examples show that overfitting occurs when BNs become more complex than allowed by supporting data and overfitting incurs computational costs as well as causing a reduction in prediction skill. CVNetica evaluates overfitting using several complexity metrics (we used level of discretization) and its impact on performance metrics (we used skill). Highlights: Cross-validation avoids overfitting, improving predictive power of Bayesian Networks. CVNetica is a Python tool for cross-validation of Bayesian Networks. Cross-validation illustrated on a data-driven and a model emulation Bayesian Network.