Comment: Why Meta-Analyses Rarely Resolve Ideological Debates. (July 2014) - Ferguson, Christopher J.
In their meta-analysis Wood, Kressel, Joshi, and Louie (2014) argue little evidence supports shifts in mating preferences across the menstrual cycle. They imply this may represent a critical weakness of evolutionary psychology theories of mating preferences more generally. This report represents a fairly common use of meta-analysis: to assemble data to support or reject a particular proposition over which there is debate. Yet, rarely do meta-analyses succeed at resolving ideological debates. Multiple decision points related to the selection, coding, effect size extraction, and interpretation of studies leaves considerable room for meta-analytic authors to interject their own beliefs. Meta-analyses are typically hailed by those who agreed a priori with their conclusion, and rejected as fatally flawed by those in disagreement. As such, meta-analyses have failed in replacing narrative reviews as more objective.