A Cautionary Note on Estimating Effect Size. Issue 1 (March 2021)

• Record Type: Journal Article
• Authors: van den Bergh, Don; Haaf, Julia M.; Ly, Alexander; Rouder, Jeffrey N.; Wagenmakers, Eric-Jan
• Is Part Of: Advances in methods and practices in psychological science. Volume 4:Issue 1(2021)
• Publication Date: 2021-03
• Subjects: effect size -- Bayesian estimation -- modeling -- shrinkage -- open materials; Psychology -- Periodicals; Psychology -- Research -- Periodicals; 150
• DOI: 10.1177/2515245921992035
• Languages: English
• ISSNs: 2515-2459

Abstract

An increasingly popular approach to statistical inference is to focus on the estimation of effect size. Yet this approach is implicitly based on the assumption that there is an effect while ignoring the null hypothesis that the effect is absent. We demonstrate how this common null-hypothesis neglect may result in effect size estimates that are overly optimistic. As an alternative to the current approach, a spike-and-slab model explicitly incorporates the plausibility of the null hypothesis into the estimation process. We illustrate the implications of this approach and provide an empirical example.