The probability of conceptual replication and the variability of effect size. (November 2019)

• Record Type: Journal Article
• Authors: Ward, Matt; Kemp, Simon
• Is Part Of: Methods in psychology. Volume 1(2019)
• Publication Date: 2019-11
• Subjects: Replication probability -- Effect size -- Variability of effect size; Psychology -- Periodicals; 150.5
• DOI: 10.1016/j.metip.2019.100002
• Languages: English
• ISSNs: 2590-2601

Abstract

Abstract: We present a model, related to an earlier model of Miller and Schwarz (2011), which shows that the probability of conceptual replication depends on the standard deviation of the true effect size, SD(δ), among other parameters. Computational analyses show that the dependency is more pronounced for smaller sample sizes. A sample of 50 meta-analyses is also investigated to obtain estimates of SD(δ), which turn out to be often in the range 0.2–0.5. Putting these two sets of results together indicates that the probability of conceptual replication is likely to vary for the range of SD(δ) often found in psychology, especially for smaller samples. The results can also be used to obtain approximate estimates of how the probability of replication is changed for typical SD(δ) in psychology.