Comparing the predictive ability of a commercial artificial intelligence early warning system with physician judgement for clinical deterioration in hospitalised general internal medicine patients: a prospective observational study. Issue 10 (10th October 2019)

• Record Type: Journal Article
• Authors: Arnold, Jonathan; Davis, Alex; Fischhoff, Baruch; Yecies, Emmanuelle; Grace, Jon; Klobuka, Andrew; Mohan, Deepika; Hanmer, Janel
• Is Part Of: BMJ open. Volume 9:Issue 10(2019)
• Publication Date: 2019-10-10
• Subjects: decision support systems -- clinical -- clinical deterioration -- clinical decision-making -- early warning systems -- health information systems; Medicine -- Research -- Periodicals; 610.72
• DOI: 10.1136/bmjopen-2019-032187
• Languages: English
• ISSNs: 2044-6055

Abstract

Abstract : Objective: Our study compares physician judgement with an automated early warning system (EWS) for predicting clinical deterioration of hospitalised general internal medicine patients. Design: Prospective observational study of clinical predictions made at the end of the daytime work-shift for an academic general internal medicine floor team compared with the risk assessment from an automated EWS collected at the same time. Setting: Internal medicine teaching wards at a single tertiary care academic medical centre in the USA. Participants: Intern physicians working on the internal medicine wards and an automated EWS (Rothman Index by PeraHealth). Outcome: Clinical deterioration within 24 hours including cardiac or pulmonary arrest, rapid response team activation or unscheduled intensive care unit transfer. Results: We collected predictions for 1874 patient days and saw 35 clinical deteriorations (1.9%). The area under the receiver operating curve (AUROC) for the EWS was 0.73 vs 0.70 for physicians (p=0.571). A linear regression model combining physician and EWS predictions had an AUROC of 0.75, outperforming physicians (p=0.016) and the EWS (p=0.05). Conclusions: There is no significant difference in the performance of the EWS and physicians in predicting clinical deterioration at 24 hours on an inpatient general medicine ward. A combined model outperformed either alone. The EWS and physicians identify partially overlapping sets of at-risk patients suggesting they rely on different cues or decision rules for their predictions. Trial registration number: NCT02648828 .