A Decision Curve Analysis to Compare Multiple TBI Models and Estimate Their Clinical Impact. (16th November 2020)

• Record Type: Journal Article
• Authors: Elahi, Cyrus; Adil, Syed M; Vissoci, Joao; Staton, Catherine; Fuller, Anthony; Haglund, Michael M; Dunn, Timothy
• Is Part Of: Neurosurgery. Volume 67(2010)Supplement 1
• Publication Date: 2020-11-16
• Subjects: Nervous system -- Surgery -- Periodicals; 617.48005
• DOI: 10.1093/neuros/nyaa447_490
• Languages: English
• ISSNs: 0148-396X

Abstract

Abstract: INTRODUCTION: Despite the potential for traumatic brain injury (TBI) prognostic models to support clinical decision making, their meaningful use for treatment decisions remains allusive. One barrier to the implementation of prognostic models is the uncertain translation of traditional model performance metrics to potential clinical impact. One alternative are decision curves, which consider a model's discrimination and claibration while quantifying the clinical impact. The clinical impact is captured as "net benefit" or the additional number of true positives identified by using the model. METHODS: Data were from a prospective TBI registry collected at a regional hospital in Moshi, Tanzania. We externally validated CRASH and IMPACT, and internally validated KCMC. We calibrated the risk predictions for all models using Platt Scaling. We used the calibrated risks to create decision curves for the entire population and for mild, moderate, and severe TBI subpopulations. We compared the net benefit between the models. The units for net benefit are true positives (i.e. patients identified as high risk by the model that are truly at increased risk for poor outcome). RESULTS: The cohort included 2972 patients. The mean admission GCS was 13.36 ± 3.12. The decision curve for all TBI severities showed KCMC modestly outperformed CRASH and IMPACT. Analysis of severity specific curves showed the three models provided similar net benefits. The exception was for moderate TBI patients where the KCMC model had the greatest net benefit. At the largest difference, KCMC provided an additional net benefit of 8 compared to CRASH and IMPACT. Put differently, the KCMC prognostic model, at a risk cut-off of 21%, identified 8 additional patients per 100 as high risk that truly had an increased risk of poor outcome. CONCLUSION: This study is the first application of decision curves to TBI, and a first attempt to quantify the potential clinical impact of a TBI prognostic model. Importantly, future work with potential end-users is paramount to identify context appropriate risk cut-offs where the tool could support triage and resource allocation.