Comparison of static and rolling logistic regression models on predicting invasive mechanical ventilation or death from COVID‐19—A retrospective, multicentre study. (21st November 2022)

• Record Type: Journal Article
• Authors: Engoren, Milo; Pancaro, Carlo; Yeldo, Nicholas S.; Kerzabi, Lotfi S.; Douville, Nicholas
• Is Part Of: Clinical respiratory journal. Volume 17:Number 1(2023)
• Page Start: 40
• Page End: 49
• Publication Date: 2022-11-21
• Subjects: COVID -- death, clinical decision support, clinical prediction models -- logistic regression -- mechanical ventilation; Respiratory organs -- Diseases -- Periodicals; Respiratory organs -- Periodicals; 616.24
• DOI: 10.1111/crj.13560
• Languages: English
• ISSNs: 1752-6981

Abstract

Abstract: Introduction: COVID‐19 virus has undergone mutations, and the introduction of vaccines and effective treatments have changed its clinical severity. We hypothesized that models that evolve may better predict invasive mechanical ventilation or death than do static models. Methods: This retrospective study of adult patients with COVID‐19 from six Michigan hospitals analysed 20 demographic, comorbid, vital sign and laboratory factors, one derived factor and nine factors representing changes in vital signs or laboratory values with time for their ability to predict death or invasive mechanical ventilation within the next 4, 8 or 24 h. Static logistic regression was constructed on the initial 300 patients and tested on the remaining 6741 patients. Rolling logistic regression was similarly constructed on the initial 300 patients, but then new patients were added, and older patients removed. Each new construction model was subsequently tested on the next patient. Static and rolling models were compared with receiver operator characteristic and precision‐recall curves. Results: Of the 7041 patients, 534 (7.6%) required invasive mechanical ventilation or died within 14 days of arrival. Rolling models improved discrimination (0.865 ± 0.010, 0.856 ± 0.007 and 0.843 ± 0.005 for the 4, 8 and 24‐h models, respectively; all p < 0.001 compared with the static logistic regressions with 0.827 ± 0.011, 0.794 ± 0.012 and 0.735 ± 0.012, respectively). Similarly, the areas under the precision‐recall curves improved from 0.006, 0.010 and 0.021 with the static models to 0.030, 0.045 and 0.076 for the 4‐, 8‐ and 24‐h rolling models, respectively, all p < 0.001. Conclusion: Rolling models with contemporaneous data maintained better metrics of performance than static models, which used older data. Abstract : When a disease or treatment is rapidly changing, traditional (static) predictive models may lose their accuracy. We show that in 7041 patients of whom 534 (7.6%) required invasive mechanical ventilation or died within 14 days that rolling logistic regression models improved both discrimination and the areas under the precision‐recall curves—all p < 0.001.