Detecting Selection Bias in Meta-Analyses with Multiple Outcomes: A Simulation Study. Issue 1 (2nd January 2021)

• Record Type: Journal Article
• Authors: Fernández-Castilla, Belén; Declercq, Lies; Jamshidi, Laleh; Beretvas, S. Natasha; Onghena, Patrick; Van den Noortgate, Wim
• Is Part Of: Journal of experimental education. Volume 89:Issue 1(2021)
• Page Start: 125
• Page End: 144
• Publication Date: 2021-01-02
• Subjects: Meta-analysis -- multiple effect sizes -- publication bias -- selective outcome reporting bias -- simulation study; Education -- Experimental methods -- Periodicals; Educational psychology -- Periodicals; Education -- Periodicals; Psychology, Experimental -- Periodicals; 370.15
• DOI: 10.1080/00220973.2019.1582470
• Languages: English
• ISSNs: 0022-0973

Abstract

Abstract: This study explores the performance of classical methods for detecting publication bias—namely, Egger's regression test, Funnel Plot test, Begg's Rank Correlation and Trim and Fill method—in meta-analysis of studies that report multiple effects. Publication bias, outcome reporting bias, and a combination of these were generated. Egger's regression test and the Funnel Plot test were extended to three-level models, and possible cutoffs for the estimator of the Trim and Fill method were explored. Furthermore, we checked whether the combination of results of several methods yielded a better control of Type I error rates. Results show that no method works well across all conditions and that performance depends mainly on the population effect size value and the total variance.