Avoid Cohen's 'Small', 'Medium', and 'Large' for Power Analysis. Issue 3 (March 2020)

• Record Type: Journal Article
• Authors: Correll, Joshua; Mellinger, Christopher; McClelland, Gary H.; Judd, Charles M.
• Is Part Of: Trends in cognitive sciences. Volume 24:Issue 3(2020)
• Page Start: 200
• Page End: 207
• Publication Date: 2020-03
• Subjects: effect size -- research methods -- research design; Cognitive science -- Periodicals; Cognitive neuroscience -- Periodicals; 153.05
• DOI: 10.1016/j.tics.2019.12.009
• Languages: English
• ISSNs: 1364-6613

Abstract

Abstract : One of the most difficult and important decisions in power analysis involves specifying an effect size. Researchers frequently employ definitions of small, medium, and large that were proposed by Jacob Cohen. These definitions are problematic for two reasons. First, they are arbitrary, based on non-scientific criteria. Second, they are inconsistent, changing dramatically and illogically as a function of the statistical test a researcher plans to use (e.g., t -test versus regression). These problems may be unknown to many researchers, but they have a huge impact on power analyses. Estimates of the required n may be inappropriately doubled or cut in half. For power analyses to have any meaning, these definitions of effect size should be avoided. Highlights: Cohen's definitions of small, medium, and large effect sizes are arbitrary and inconsistent. Illogical variation in these definitions has a profound and largely unappreciated impact on power analyses. Researchers may inadvertently and incorrectly conclude that they need samples that are twice as large as necessary, or only half as large as necessary, to achieve a given power. These definitions of effect size should not be used for power analysis. A common metric for effect size would facilitate the field's ability to think meaningfully about power.