A time-incorporated SOFA score-based machine learning model for predicting mortality in critically ill patients: A multicenter, real-world study. (July 2022)

• Record Type: Journal Article
• Authors: Liu, Yang; Gao, Kun; Deng, Hongbin; Ling, Tong; Lin, Jiajia; Yu, Xianqiang; Bo, Xiangwei; Zhou, Jing; Gao, Lin; Wang, Peng; Hu, Jiajun; Zhang, Jian; Tong, Zhihui; Liu, Yuxiu; Shi, Yinghuan; Ke, Lu; Gao, Yang; Li, Weiqin
• Is Part Of: International journal of medical informatics. Volume 163(2022)
• Publication Date: 2022-07
• Subjects: Organ dysfunction -- SOFA -- Time dimension -- Machine learning -- Mortality prediction; AUROC Area Under the receiver operating characteristic Curve -- eICU-CRD eICU Collaborative Research Database -- ICU Intensive Care Unit -- LIME Local Interpretable Model-Agnostic Explanations -- LR Logistic Regression -- MIMIC Medical Information Mart for Intensive Care -- OATI Organ dysfunction Aggravated Time Index -- OD Organ Dysfunction -- OFTI Organ dysFunction Time Index -- OUTI Organ dysfunction Unalleviated Time Index -- RASS Richmond Agitation-Sedation Scale -- RF Random Forest -- SHAP SHapley Additive exPlanations -- SOFA Sequence Organ Failure Assessment -- STAUC SOFA-time Area Under Curve -- SVM Support Vector Machine -- XGBoost eXtreme Gradient Boosting; Medical informatics -- Periodicals; Information science -- Periodicals; Computers -- Periodicals; Medical technology -- Periodicals; Medical Informatics -- Periodicals; Technology, Medical -- Periodicals; Computers; Information science; Medical informatics; Medical technology; Electronic journals; Periodicals; Electronic journals; 610.285
• DOI: 10.1016/j.ijmedinf.2022.104776
• Languages: English
• ISSNs: 1386-5056

Abstract

Highlight: The best performance was achieved by the XGBoost algorithm based on the time dimension of organ dysfunction with AUC > 0.8. Time features such as the organ dysfunction unalleviated time Index are important for mortality prediction. The explainable model could assist clinicians in risk stratification and personalized treatment decision-making for critically ill patients. Abstract: Background: Organ dysfunction (OD) assessment is essential in intensive care units (ICUs). However, current OD assessment scores merely describe the number and the severity of each OD, without evaluating the duration of organ injury. The objective of this study is to develop and validate a machine learning model based on the Sequential Organ Failure Assessment (SOFA) score for the prediction of mortality in critically ill patients. Material and methods: Data from the eICU Collaborative Research Database and Medical Information Mart for Intensive Care (MIMIC) -III were mixed for model development. The MIMIC-IV and Nanjing Jinling Hospital Surgical ICU database were used as external test set A and set B, respectively. The outcome of interest was in-ICU mortality. A modified SOFA model incorporating time-dimension (T-SOFA) was stepwise developed to predict ICU mortality using extreme gradient boosting (XGBoost), support vector machine, random forest and logistic regression algorithms. Time-dimensional features were calculated based on six consecutive SOFA scores collected every 12 h within the first three days of admission. The predictive performance was assessed with the area under the receiver operating characteristic curves (AUROC) and calibration plot. Results: A total of 82, 132 patients from the real-world datasets were included in this study, and 7, 494 patients (9.12%) died during their ICU stay. The T-SOFA M3 that incorporated the time-dimension features and age, using the XGBoost algorithm, significantly outperformed the original SOFA score in the validation set (AUROC 0.800 95% CI [0.787–0.813] vs. 0.693 95% CI [0.678–0.709], p < 0.01). Good discrimination and calibration were maintained in the test set A and B, with AUROC of 0.803, 95% CI [0.791–0.815] and 0.830, 95% CI [0.789–0.870], respectively. Conclusions: The time-incorporated T-SOFA model could significantly improve the prediction performance of the original SOFA score and is of potential for identifying high-risk patients in future clinical application.