Meaningful Model Comparisons have to Include Reasonable Competing Models and also all Data: A Rejoinder to Rieskamp (2018). (1st March 2018)

• Record Type: Journal Article
• Authors: Jekel, Marc; Glöckner, Andreas
• Other Names: Marewski Julian N. guestEditor.; Bröder Arndt guestEditor.; Glöckner Andreas guestEditor.
• Is Part Of: Journal of behavioral decision making. Volume 31:Number 2(2018)
• Page Start: 289
• Page End: 293
• Publication Date: 2018-03-01
• Subjects: model comparison -- probabilistic inferences -- strategy selection -- decision making -- adaptive toolbox -- fast‐and‐frugal heuristics; Decision making -- Periodicals; Managerial economics -- Periodicals; Behavior -- Periodicals; Decision Making -- Periodicals; Prise de décision -- Périodiques; Économie d'entreprise -- Périodiques; 153.83
• DOI: 10.1002/bdm.2076
• Languages: English
• ISSNs: 0894-3257

Abstract

Abstract: Proper methods for model comparisons require including reasonable models and all data. In an article in this issue, we highlight that excluding some models was common in parts of the literature supposedly supporting fast‐and‐frugal heuristics and show in comprehensive simulations and exemplified reanalyses of studies by Rieskamp that this can–under specific conditions–lead to wrong conclusions. In a comment in this issue, Rieskamp provides no conclusive arguments against these basic principles of proper methodology. Hence, the principles to include (i) all reasonable competing models and (ii) all data still stand and have to remain the standards for evaluating past and future empirical work. Rieskamp addresses the first of the two issues in a Bayesian analysis. He, however, violates the second principle by omitting data to be able to uphold his previous conclusion, which invalidates his results. We explain in more detail why exclusion of data leads to biased estimates in this context and provide a refined Bayesian analysis–of course based on all data–that underlines our previously raised concerns about the criticized studies. Instead of arguing against the two basic principles of model comparison, we recommend authors of affected studies to conduct proper reanalyses including all models and data. Copyright © 2018 John Wiley & Sons, Ltd.