A comprehensive review of reporting practices in psychological journals: Are effect sizes really enough?. (February 2013)

• Record Type: Journal Article
• Authors: Fritz, Astrid; Scherndl, Thomas; Kühberger, Anton
• Is Part Of: Theory & psychology. Volume 23:Number 1(2013)
• Page Start: 98
• Page End: 122
• Publication Date: 2013-02
• Subjects: confidence interval -- effect size -- NHST -- publication bias -- significance testing -- statistical power; Psychology -- Periodicals; 150.1
• DOI: 10.1177/0959354312436870
• Languages: English
• ISSNs: 0959-3543

Abstract

Over-reliance on significance testing has been heavily criticized in psychology. Therefore the American Psychological Association recommended supplementing the p value with additional elements such as effect sizes, confidence intervals, and considering statistical power seriously. This article elaborates the conclusions that can be drawn when these measures accompany the p value. An analysis of over 30 summary papers (including over 6, 000 articles) reveals that, if at all, only effect sizes are reported in addition to p 's (38%). Only every 10th article provides a confidence interval and statistical power is reported in only 3% of articles. An increase in reporting frequency of the supplements to p 's over time owing to stricter guidelines was found for effect sizes only. Given these practices, research faces a serious problem in the context of dichotomous statistical decision making: since significant results have a higher probability of being published (publication bias), effect sizes reported in articles may be seriously overestimated.