A Comparative Assessment of Adverse Event Classification in the Out-of-hospital Setting. (2nd October 2014)

• Record Type: Journal Article
• Authors: Patterson, P. Daniel; Lave, Judith R.; Weaver, Matthew D.; Guyette, Francis X.; Arnold, Robert M.; Martin-Gill, Christian; Rittenberger, Jon C.; Krackhardt, David; Mosesso, Vincent N.; Roth, Ronald N.; Wadas, Richard J.; Yealy, Donald M.
• Is Part Of: Prehospital emergency care. Volume 18:Number 4(2014:Oct./Dec.)
• Page Start: 495
• Page End: 504
• Publication Date: 2014-10-02
• Subjects: 362.18
• DOI: 10.3109/10903127.2014.916022
• Languages: English
• ISSNs: 1090-3127

Abstract

<abstract> <title>Abstract</title> <p> <bold>Objectives.</bold> We sought to test reliability of two approaches to classify adverse events (AEs) associated with helicopter EMS (HEMS) transport. <bold>Methods.</bold> The first approach for AE classification involved flight nurses and paramedics (RN/Medics) and mid-career emergency physicians (MC-EMPs) independently reviewing 50 randomly selected HEMS medical records. The second approach involved RN/Medics and MC-EMPs meeting as a group to openly discuss 20 additional medical records and reach consensus-based AE decision. We compared all AE decisions to a reference criterion based on the decision of three senior emergency physicians (Sr-EMPs). We designed a study to detect an improvement in agreement (reliability) from fair (kappa = 0.2) to moderate (kappa = 0.5). We calculated sensitivity, specificity, percent agreement, and positive and negative predictive values (PPV/NPV). <bold>Results.</bold> For the independent reviews, the Sr-EMP group identified 26 AEs while individual clinician reviewers identified between 19 and 50 AEs. Agreement on the presence/absence of an AE between Sr-EMPs and three MC-EMPs ranged from κ = 0.20 to κ = 0.25. Agreement between Sr-EMPs and three RN/Medics ranged from κ = 0.11 to κ = 0.19. For the consensus/open-discussion approach, the Sr-EMPs identified 13 AEs, the MC-EMP group identified 18 AEs, and RN/medic group identified 36 AEs. Agreement between Sr-EMPs and MC-EMP group was (κ = 0.30 95%CI −0.12, 0.72), whereas agreement between Sr-EMPs and RN/medic group was (κ = 0.40 95%CI 0.01, 0.79). Agreement between all three groups was fair (κ = 0.33, 95%CI 0.06, 0.66). Percent agreement (58–68%) and NPV (63–76%) was moderately dissimilar between clinicians, while sensitivity (25–80%), specificity (43–97%), and PPV (48–83%) varied. <bold>Conclusions.</bold> We identified a higher level of agreement/reliability in AE decisions utilizing a consensus-based approach for review rather than independent reviews.</p> </abstract>