Problems Caused by Heterogeneity in Meta-Analysis: A Case Study of Acupuncture Trials. Issue 1 (February 2014)

• Record Type: Journal Article
• Authors: Prady, Stephanie L; Burch, Jane; Crouch, Simon; MacPherson, Hugh
• Is Part Of: Acupuncture in medicine. Volume 32:Issue 1(2014)
• Page Start: 56
• Page End: 61
• Publication Date: 2014-02
• Subjects: Acupuncture -- Periodicals; 615.892
• DOI: 10.1136/acupmed-2013-010364
• Languages: English
• ISSNs: 0964-5284

Abstract

Objectives: To illustrate the pitfalls of using meta-analysis to combine estimates of effect in trials that are highly varied and have a high potential for bias. Methods: We used a random-effects meta-analysis to pool the results of 51 sham-controlled acupuncture trials of chronic pain published in English before 2008 and explored the heterogeneity using meta-regression. We repeated the process on a subset of these trials that used a visually credible non-penetrating sham device as control (N = 12). Results: In both analyses there were high levels of heterogeneity and many studies were at risk from potential bias. The heterogeneity was not explained by meta-regression. Conclusions: Trials of interventions that have high potential for bias, such as many in the acupuncture literature, do not meet the assumptions of the statistical procedure that underlie random-effects meta-analysis. Even in the absence of bias, heterogeneity in meta-analyses is not accounted for by the CIs around the pooled estimate.