The transparency and reproducibility of systematic reviews in forensic science. (November 2022)

• Record Type: Journal Article
• Authors: Chin, Jason M.; Growns, Bethany; Sebastian, Joel; Page, Matthew J.; Nakagawa, Shinichi
• Is Part Of: Forensic science international. Volume 340(2022)
• Publication Date: 2022-11
• Subjects: Forensic science -- Systematic reviews -- Meta-analyses -- Metaresearch -- Open science -- Expert evidence; Medical jurisprudence -- Periodicals; Chemistry, Forensic -- Periodicals; Forensic Medicine -- Periodicals; Médecine légale -- Périodiques; Chimie légale -- Périodiques; Gerechtelijke geneeskunde; Gerechtelijke chemie; Gerechtelijke psychiatrie; Chemistry, Forensic; Medical jurisprudence; Electronic journals; Periodicals; Electronic journals; 614.1
• DOI: 10.1016/j.forsciint.2022.111472
• Languages: English
• ISSNs: 0379-0738

Abstract

Abstract: Systematic reviews are indispensable tools for both reliably informing decision-makers about the state of the field and for identifying areas that need further study. Their value, however, depends on their transparency and reproducibility. Readers should be able to determine what was searched for and when, where the authors searched, and whether that search was predetermined or evolved based on what was found. In this article, we measured the transparency and reproducibility of systematic reviews in forensic science, a field where courts, policymakers, and legislators count on systematic reviews to make informed decisions. In a sample of 100 systematic reviews published between 2018 and 2021, we found that completeness of reporting varied markedly. For instance, 50 % of reviews claimed to follow a reporting guideline and such statements were only modestly related to compliance with that reporting guideline. As to specific reporting items, 82 % reported all of the databases searched, 22 % reported the review's full Boolean search logic, and just 7 % reported the review was registered. Among meta-analyses (n = 23), only one stated data was available and none stated the analytic code was available. After considering the results, we end with recommendations for improved regulation of reporting practices, especially among journals. Our results may serve as a useful benchmark as the field evolves. Highlights: Systematic reviews assist decision-makers and researchers in determining what is known and unknown about a field. We audited the transparency and reproducibility 100 forensic science systematic reviews published from 2018 to 2021. Half of reviews purported to follow the PRISMA reporting guidelines, but these statements were only modestly related to compliance. Transparency of reporting was uneven. Forensic science journals should more proactively regulate the systematic reviews they publish.