Explainable deep learning model to predict invasive bacterial infection in febrile young infants: A retrospective study. (April 2023)

• Record Type: Journal Article
• Authors: Yang, Ying; Wang, Yi-Min; Lin, Chun-Hung Richard; Cheng, Chi-Yung; Tsai, Chi-Ming; Huang, Ying-Hsien; Chen, Tien-Yu; Chiu, I-Min
• Is Part Of: International journal of medical informatics. Volume 172(2023)
• Publication Date: 2023-04
• Subjects: Explainable machine learning -- Invasive bacterial infection -- Febrile young infant -- Emergency department; Medical informatics -- Periodicals; Information science -- Periodicals; Computers -- Periodicals; Medical technology -- Periodicals; Medical Informatics -- Periodicals; Technology, Medical -- Periodicals; Computers; Information science; Medical informatics; Medical technology; Electronic journals; Periodicals; Electronic journals; 610.285
• DOI: 10.1016/j.ijmedinf.2023.105007
• Languages: English
• ISSNs: 1386-5056

Abstract

Highlights: The explainable deep learning model exhibits better prediction performance compared with previous scoring systems. Utilizing the explainable deep learning model, clinicians can better stratify low-risk patients and understand why the model makes certain predictions. Such explainable prediction model can be a reference and support clinicians during decision-making. Abstract: Background: Machine learning models have demonstrated superior performance in predicting invasive bacterial infection (IBI) in febrile infants compared to commonly used risk stratification criteria in recent studies. However, the black-box nature of these models can make them difficult to apply in clinical practice. In this study, we developed and validated an explainable deep learning model that can predict IBI in febrile infants ≤ 60 days of age visiting the emergency department. Methods: We conducted a retrospective study of febrile infants aged ≤ 60 days who presented to the pediatric emergency department of a medical center in Taiwan between January 1, 2011 and December 31, 2019. Patients with uncertain test results and complex chronic health conditions were excluded. IBI was defined as the growth of a pathogen in the blood or cerebrospinal fluid. We used a deep neural network to develop a predictive model for IBI and compared its performance to the IBI score and step-by-step approach. The SHapley Additive Explanations (SHAP) technique was used to explain the model's predictions at different levels. Results: Our study included 1847 patients, 53 (2.7%) of whom had IBI. The deep learning model performed similarly to the IBI score and step-by-step approach in terms of sensitivity and negative predictive value, but provided better specificity (54%), positive predictive value (5%), and area under the receiver-operating characteristic curve (0.87). SHapley Additive exPlanations identified five influential predictive variables (absolute neutrophil count, body temperature, heart rate, age, and C-reactive protein). Conclusion: We have developed an explainable deep learning model that can predict IBI in febrile infants aged 0–60 days. The model not only performs better than previous scoring systems, but also provides insight into how it arrives at its predictions through individual features and cases.