3* Evaluation of an Artificial Intelligence (AI) system to augment clinical risk prediction of Trauma Induced Coagulopathy in the pre-hospital setting: a prospective observational study. Issue 5 (26th September 2022)

• Record Type: Journal Article
• Authors: Marsden, Max; Perkins, Zane; Marsh, William; Christian, Michael; Lyon, Richard; Davenport, Ross; Tai, Nigel
• Is Part Of: BMJ military health. Volume 168:Issue 5(2022)
• Page Start: e1
• Page End: e1
• Publication Date: 2022-09-26
• Subjects: Medicine, Military -- Periodicals; Military hygiene -- Periodicals; 355.345
• DOI: 10.1136/bmjmilitary-2022-RSMabstracts.3
• Languages: English
• ISSNs: 2633-3767

Abstract

Abstract : Background: The potential of AI systems to support pre-hospital clinical decision-making in both military and civilian settings is significant. However, whilst such algorithms are increasingly available, far less attention has been paid to understanding the impact of such systems on clinical performance. This study had two aims; first, to compare the performance of expert clinicians against an AI system in a real-world clinical setting and second, to assess the impact of augmenting expert clinical prediction with an AI system. Methods: Two civilian UK Air Ambulances were selected as surrogate settings relevant to military practice. We performed a prospective study over a six-month period where expert pre-hospital clinicians' judgement of the risk of Trauma Induced Coagulopathy (TIC) in injured patients was assessed and compared to the performance of an AI system. Two TIC risk predictions were generated for every patient: an AI prediction and a human prediction. Measures of predictive performance included discrimination, calibration, and overall accuracy. Results: Overall, 51 expert clinicians were enrolled in the study providing 184 patient interactions for analysis. The studied patients had a median age of 31 (range 16, 89), median injury severity score of 17 (IQR 9, 34), 75% were male, and 19% developed TIC. Aim 1: The AI system performed better than clinicians; higher discrimination [AUROC 0.87 (0.79, 0.95) versus 0.83 (0.74, 0.92)] better calibration [0.37 (-0.14, 0.89) versus -1.19 (-1.73, -0.65)] and more accurate [Brier Skill Score 0.34 (0.19, 0.48) versus 0.00 (-0.41, 0.30)]. Aim 2: Risk prediction was better in all performance metrics when clinicians were assisted with the AI system [AUROC 0.88 (0.80, 0.95) versus 0.83 (0.74, 0.92)] Conclusions: AI systems can improve human risk prediction in the pre-hospital setting. In the military environment, where austerity and lack of senior clinical expertise may affect outcomes, the benefit of implementing predictive AI should be substantial. (*awarded First Place)