92 Evaluation of an Artificial Intelligence (AI) System to Augment Clinical Risk Prediction of Trauma Induced Coagulopathy: A Prospective Observational Study. (28th February 2022)

• Record Type: Journal Article
• Authors: Marsden, M.; Perkins, Z.; Marsh, W.; Christian, M.; Lyon, R.; Davenport, R.; Tai, N.
• Is Part Of: British journal of surgery. Volume 109(2022)Supplement 1
• Publication Date: 2022-02-28
• Subjects: Surgery -- Periodicals; 617.005
• DOI: 10.1093/bjs/znac041.005
• Languages: English
• ISSNs: 0007-1323

Abstract

Abstract: Introduction: The potential of AI systems to support pre-hospital clinical decision-making is significant. However, whilst such algorithms are increasingly available, far less attention has been paid to understanding the impact of such systems on clinical performance. This study had two aims; first, to compare the performance of expert clinicians against an AI system in a real-world clinical setting and second, to assess the impact of augmenting expert clinical prediction with an AI system. Method: We performed a prospective study at two UK Air Ambulances services over a six-month period. Expert pre-hospital clinicians' judgement of the risk of Trauma Induced Coagulopathy (TIC) in injured patients was assessed and compared to the performance of an AI system. Two TIC risk predictions were generated for every patient: an AI prediction and a human prediction. Results: Overall, 51 expert clinicians were enrolled in the study providing 184 patient interactions for analysis. Aim 1: The AI system performed better than clinicians; higher discrimination [AUROC 0.87 (0.79, 0.95) versus 0.83 (0.74, 0.92)] better calibration [0.37 (-0.14, 0.89) versus -1.19 (-1.73, -0.65)] and more accurate [Brier Skill Score 0.34 (0.19, 0.48) versus 0.00 (-0.41, 0.30)]. Aim 2: Risk prediction was better in all performance metrics when clinicians were assisted with the AI system [AUROC 0.88 (0.80, 0.95) versus 0.83 (0.74, 0.92)] Conclusions: AI systems can improve human risk prediction in the pre-hospital setting. In settings of low resources where a lack of senior clinical expertise may affect outcomes, the benefit of implementing predictive AI is substantial.