An assessment of statistical methods for nonindependent data in ecological meta‐analyses. Issue 12 (7th October 2020)

• Record Type: Journal Article
• Authors: Song, Chao; Peacor, Scott D.; Osenberg, Craig W.; Bence, James R.
• Is Part Of: Ecology. Volume 101:Issue 12(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-10-07
• Subjects: meta‐analysis -- nonindependence -- pseudoreplication -- random effect -- hierarchical model -- robust variance estimation; Ecology -- Periodicals; Ecology -- Periodicals; Écologie -- Périodiques; Ecologie; Écologie; Écologie animale; Écologie végétale; Ecology; Periodicals; 577.05
• DOI: 10.1002/ecy.3184
• Languages: English
• ISSNs: 0012-9658

Abstract

Abstract: In ecological meta‐analyses, nonindependence among observed effect sizes from the same source paper is common. If not accounted for, nonindependence can seriously undermine inferences. We compared the performance of four meta‐analysis methods that attempt to address such nonindependence and the standard random‐effect model that ignores nonindependence. We simulated data with various types of within‐paper nonindependence, and assessed the standard deviation of the estimated mean effect size and Type I error rate of each method. Although all four methods performed substantially better than the standard random‐effects model that assumes independence, there were differences in performance among the methods. A two‐step method that first summarizes the multiple observed effect sizes per paper using a weighted mean and then analyzes the reduced data in a standard random‐effects model, and a robust variance estimation method performed consistently well. A hierarchical model with both random paper and study effects gave precise estimates but had a higher Type I error rates, possibly reflecting limitations of currently available meta‐analysis software. Overall, we advocate the use of the two‐step method with a weighted paper mean and the robust variance estimation method as reliable ways to handle within‐paper nonindependence in ecological meta‐analyses.