A Primer for Model Selection: The Decisive Role of Model Complexity. Issue 3 (10th March 2018)

• Record Type: Journal Article
• Authors: Höge, Marvin; Wöhling, Thomas; Nowak, Wolfgang
• Is Part Of: Water resources research. Volume 54:Issue 3(2018)
• Page Start: 1688
• Page End: 1715
• Publication Date: 2018-03-10
• Subjects: model selection -- model complexity -- information criteria (IC) -- primer; Hydrology -- Periodicals; 333.91
• DOI: 10.1002/2017WR021902
• Languages: English
• ISSNs: 0043-1397

Abstract

Abstract: Selecting a "best" model among several competing candidate models poses an often encountered problem in water resources modeling (and other disciplines which employ models). For a modeler, the best model fulfills a certain purpose best (e.g., flood prediction), which is typically assessed by comparing model simulations to data (e.g., stream flow). Model selection methods find the "best" trade‐off between good fit with data and model complexity. In this context, the interpretations of model complexity implied by different model selection methods are crucial, because they represent different underlying goals of modeling. Over the last decades, numerous model selection criteria have been proposed, but modelers who primarily want to apply a model selection criterion often face a lack of guidance for choosing the right criterion that matches their goal. We propose a classification scheme for model selection criteria that helps to find the right criterion for a specific goal, i.e., which employs the correct complexity interpretation. We identify four model selection classes which seek to achieve high predictive density, low predictive error, high model probability, or shortest compression of data. These goals can be achieved by following either nonconsistent or consistent model selection and by either incorporating a Bayesian parameter prior or not. We allocate commonly used criteria to these four classes, analyze how they represent model complexity and what this means for the model selection task. Finally, we provide guidance on choosing the right type of criteria for specific model selection tasks. (A quick guide through all key points is given at the end of the introduction.) Key Points: Model selection criteria are often chosen arbitrarily; we offer a guiding classification system for commonly used criteria centered around their representation of model complexity The classification considers underlying definitions of model complexity which encompass different foci on identifying versus approaching an underlying truth, conducted in an either Bayesian or non‐Bayesian way Each model selection class pursues a specific goal; we outline which one is most suitable for a specific modeling task