Exploring consequences of simulation design for apparent performance of methods of meta-analysis. (July 2021)

• Record Type: Journal Article
• Authors: Kulinskaya, Elena; Hoaglin, David C.; Bakbergenuly, Ilyas
• Is Part Of: Statistical methods in medical research. Volume 30:Number 7(2021)
• Page Start: 1667
• Page End: 1690
• Publication Date: 2021-07
• Subjects: Meta-analysis -- odds-ratio -- random-effects model -- random probabilities -- random sample sizes; Medicine -- Research -- Statistical methods -- Periodicals; Research -- Periodicals; Review Literature -- Periodicals; Statistics -- methods -- Periodicals; Médecine -- Recherche -- Méthodes statistiques -- Périodiques; 610.727
• DOI: 10.1177/09622802211013065
• Languages: English
• ISSNs: 0962-2802

Abstract

Contemporary statistical publications rely on simulation to evaluate performance of new methods and compare them with established methods. In the context of random-effects meta-analysis of log-odds-ratios, we investigate how choices in generating data affect such conclusions. The choices we study include the overall log-odds-ratio, the distribution of probabilities in the control arm, and the distribution of study-level sample sizes. We retain the customary normal distribution of study-level effects. To examine the impact of the components of simulations, we assess the performance of the best available inverse–variance–weighted two-stage method, a two-stage method with constant sample-size-based weights, and two generalized linear mixed models. The results show no important differences between fixed and random sample sizes. In contrast, we found differences among data-generation models in estimation of heterogeneity variance and overall log-odds-ratio. This sensitivity to design poses challenges for use of simulation in choosing methods of meta-analysis.