"A little learning is a dangerous thing": A call for better understanding of the term 'systematic review'. (February 2017)

• Record Type: Journal Article
• Authors: Haddaway, Neal R; Land, Magnus; Macura, Biljana
• Is Part Of: Environment international. Volume 99(2017:Feb.)
• Page Start: 356
• Page End: 360
• Publication Date: 2017-02
• Subjects: Evidence-based policy -- Evidence -- Evidence synthesis -- Grey literature -- Risk of bias -- Science policy; Environmental protection -- Periodicals; Environmental health -- Periodicals; Environmental monitoring -- Periodicals; Environmental Monitoring -- Periodicals; Environnement -- Protection -- Périodiques; Hygiène du milieu -- Périodiques; Environnement -- Surveillance -- Périodiques; Environmental health; Environmental monitoring; Environmental protection; Periodicals; 333.705
• DOI: 10.1016/j.envint.2016.12.020
• Languages: English
• ISSNs: 0160-4120

Abstract

Abstract: Systematic reviews are becoming a widely accepted gold standard in evidence synthesis for evidence-based and –informed policy and practice. Many organisations exist to coordinate the registration, conduct and publication of systematic reviews across a range of disciplines, including medicine, international development, and environmental management and biodiversity conservation. As the term 'systematic review' becomes more widely recognised, however, there is a risk that stakeholders may have only partial understanding of the rigorous methods required to produce a reliable systematic review. Here, we highlight one such example from the field of education and international development, where a World Bank report claimed to 'systematically review' six 'systematic reviews' that found divergent results. We critically appraise the six included reviews and the World Bank report itself using an a priori quality assessment tool. Our analysis shows that none of the six included reviews are classifiable as systematic reviews according to widely accepted criteria. We also find that the World Bank report failed to use true systematic review methods to synthesise the included reviews findings. Our study demonstrates the risks associated with partial understanding of the added value associated with systematic reviews and highlights a need for improved awareness of what systematic reviews are. Highlights: Systematic reviews (SR) are rapidly becoming 'gold standards' in evidence synthesis. Misunderstandings exist over what is required to make a review systematic. We highlight a recent World Bank report that highlights such misunderstandings. The report incorrectly recognised ordinary reviews as being systematic reviews. We critically appraise the report and highlight how this mistake should be avoided.