Examining Psychological Science Through Systematic Meta-Method Analysis: A Call for Research. Issue 4 (December 2019)

• Record Type: Journal Article
• Authors: Elson, Malte
• Is Part Of: Advances in methods and practices in psychological science. Volume 2:Issue 4(2019)
• Page Start: 350
• Page End: 363
• Publication Date: 2019-12
• Subjects: meta-analysis -- metascience -- methodology -- standardization -- synthesis -- open data; Psychology -- Periodicals; Psychology -- Research -- Periodicals; 150
• DOI: 10.1177/2515245919863296
• Languages: English
• ISSNs: 2515-2459

Abstract

Research synthesis is based on the assumption that when the same association between constructs is observed repeatedly in a field, the relationship is probably real, even if its exact magnitude can be debated. Yet the probability that the relationship is real is a function not only of recurring results, but also of the quality and consistency of the empirical procedures that produced those results and that any meta-analysis necessarily inherits. Standardized protocols in data collection, analysis, and interpretation are foundations of empiricism and a healthy sign of a discipline's maturity. I propose that meta-analysis as typically applied in psychology will benefit from complementing aggregation of observed effect sizes with systematic examination of the standardization of the methodology that deterministically produced them. I describe potential units of analysis and offer two examples illustrating the benefits of such efforts. Ideally, this synergetic approach will advance theory by improving the quality of meta-analytic inferences.