Assessing Strength of Evidence of Appropriate Use Criteria for Diagnostic Imaging Examinations. (17th January 2016)

• Record Type: Journal Article
• Authors: Lacson, Ronilda; Raja, Ali S; Osterbur, David; Ip, Ivan; Schneider, Louise; Bain, Paul; Mita, Carol; Whelan, Julia; Silveira, Patricia; Dement, David; Khorasani, Ramin
• Is Part Of: Journal of the American Medical Informatics Association. Volume 23:Number 3(2016:May)
• Page Start: 649
• Page End: 653
• Publication Date: 2016-01-17
• Subjects: Clinical decision support system -- clinical practice guidelines -- evidence-based practice -- diagnostic imaging; Medical informatics -- Periodicals; Information Services -- Periodicals; Medical Informatics -- Periodicals; Médecine -- Informatique -- Périodiques; Informatica; Geneeskunde; Informatique médicale; Computer network resources; Electronic journals; 610.285
• DOI: 10.1093/jamia/ocv194
• Languages: English
• ISSNs: 1067-5027

Abstract

Abstract : Objective For health information technology tools to fully inform evidence-based decisions, recommendations must be reliably assessed for quality and strength of evidence. We aimed to create an annotation framework for grading recommendations regarding appropriate use of diagnostic imaging examinations. Methods The annotation framework was created by an expert panel (clinicians in three medical specialties, medical librarians, and biomedical scientists) who developed a process for achieving consensus in assessing recommendations, and evaluated by measuring agreement in grading the strength of evidence for 120 empirically selected recommendations using the Oxford Levels of Evidence. Results Eighty-two percent of recommendations were assigned to Level 5 (expert opinion). Inter-annotator agreement was 0.70 on initial grading (κ = 0.35, 95% CI, 0.23-0.48). After systematic discussion utilizing the annotation framework, agreement increased significantly to 0.97 (κ = 0.88, 95% CI, 0.77-0.99). Conclusions A novel annotation framework was effective for grading the strength of evidence supporting appropriate use criteria for diagnostic imaging exams.