Calibrated Bayes Factors Should Not Be Used: A Reply to Hoijtink, van Kooten, and Hulsker. (2nd January 2016)

• Record Type: Journal Article
• Authors: Morey, Richard D.; Wagenmakers, Eric-Jan; Rouder, Jeffrey N.
• Is Part Of: Multivariate behavioral research. Volume 51:Number 1(2016)
• Page Start: 11
• Page End: 19
• Publication Date: 2016-01-02
• Subjects: Statistical inference; Bayesian statistics; Bayes factors; error rates; Psychometrics -- Periodicals; Psychology, Experimental -- Periodicals; Psychology, Experimental; Psychometrics; Periodicals; 150.15195
• DOI: 10.1080/00273171.2015.1052710
• Languages: English
• ISSNs: 0027-3171

Abstract

ABSTRACT: Hoijtink, Kooten, and Hulsker (2016 ) present a method for choosing the prior distribution for an analysis with Bayes factor that is based on controlling error rates, which they advocate as an alternative to our more subjective methods (Morey & Rouder, 2014 ; Rouder, Speckman, Sun, Morey, & Iverson, 2009 ; Wagenmakers, Wetzels, Borsboom, & van der Maas, 2011 ). We show that the method they advocate amounts to a simple significance test, and that the resulting Bayes factors are not interpretable. Additionally, their method fails in common circumstances, and has the potential to yield arbitrarily high Type II error rates. After critiquing their method, we outline the position on subjectivity that underlies our advocacy of Bayes factors.