2  Does end-user-engaged peer review improve the relevance, replicability, and accessibility of research evidence?. (18th July 2022)

• Record Type: Journal Article
• Authors: Li, Edward W
• Is Part Of: BMJ evidence-based medicine. Volume 27(2022)Supplement 2
• Page Start: A13
• Page End: A13
• Publication Date: 2022-07-18
• Subjects: Evidence-based medicine -- Periodicals; 616.005
• DOI: 10.1136/ebm-2022-EBMLive.24
• Languages: English
• ISSNs: 2515-446X

Abstract

Abstract : Introduction: Current forms of peer review are characterized by scrutinization of research evidence and grant applications by experts within the same field. Despite the homogeneity in reviewers' fields of expertise, modern peer review practices continue to raise concerns about poor inter-reviewer agreement, lack of transparency, and reviewer bias in publication and funding decisions. Furthermore, scholars point to inadequacies in traditional forms of peer review for facilitating efficient dissemination of research evidence to end-users, as well as for assessing the clinical relevance of research proposals. These shortfalls are concerns to both researchers invested in quality assurance and clinicians basing their practice on published evidence. This abstract therefore presents a revised approach to increasing the relevance, replicability, and accessibility of research evidence, involving end-users themselves as active participants in an integrated peer review process for research publication and funding applications. Objectives: The purpose of this investigation is to examine the effects of end-user-engaged (EUE) peer review on relevance, replicability, and accessibility of research, based on criteria listed in the Evidence-Based Medicine (EBM) Manifesto. METHODS: A systematic search of articles indexed in MEDLINE, PubMed, and the Cochrane Library was performed using various combinations and synonyms of the search terms ('end?user*', 'public', 'community', 'open', 'interactive' AND 'peer review', 'grant review') AND ('relevan*', 'important*', 'reliabl*', 'valid*', 'bias', 'replica*', 'transparen*', 'generalizab*', 'accessib*', 'communicat*'). All articles examining an EUE peer review framework as an exposure, reporting relevant outcome(s), and written in English were included. Results: Of the articles retrieved, six were found to meet the inclusion criteria. According to these articles, variations on EUE peer review had been implemented across multiple journals, such as the British Medical Journal, Lancet, and Public Library of Science. Journal readers, including patients and clinicians, were able to preview and make comments on submitted manuscripts during an iterative review process that sought collaboration from expert reviewers and authors as well. Reported benefits of an EUE framework included high quality and impact of research evidence, low manuscript rejection rates, and high reader attractivity. One article found that inclusion of patients in National Institutes of Health research funding review panels helped to ensure that projects addressed important scientific health problems, would be conducted in appropriate environments and populations, would improve public understanding of research, and would enhance accountability between academia and communities. Analyses led by evaluation experts from the social sciences provided further evidence of the ability of multi-stage open peer review to identify high-impact submissions. Conclusions: Increased end-user participation in peer review appears to be associated with strong research accessibility, replicability, and relevance, as measured by research impact, readership, quality, and focus on relevant research problems and settings. As consistent with the EBM Manifesto, patients are placed centre-stage in EUE peer review, as both leaders and consumers of clinically important research evidence. Further research is needed to clarify whether these effects are mediated by larger numbers of reviewers or the direct involvement of end-users in EUE peer review.