Impact of diagnosis code grouping method on clinical prediction model performance: A multi-site retrospective observational study. (July 2021)

• Record Type: Journal Article
• Authors: Kansal, Aman; Gao, Michael; Balu, Suresh; Nichols, Marshall; Corey, Kristin; Kashyap, Sehj; Sendak, Mark
• Is Part Of: International journal of medical informatics. Volume 151(2021)
• Publication Date: 2021-07
• Subjects: International Classification of Disease -- Clinical classification software -- Elixhauser -- Reproducibility -- MIMIC-III; Medical informatics -- Periodicals; Information science -- Periodicals; Computers -- Periodicals; Medical technology -- Periodicals; Medical Informatics -- Periodicals; Technology, Medical -- Periodicals; Computers; Information science; Medical informatics; Medical technology; Electronic journals; Periodicals; Electronic journals; 610.285
• DOI: 10.1016/j.ijmedinf.2021.104466
• Languages: English
• ISSNs: 1386-5056

Abstract

Graphical abstract: Highlights: Methods for incorporating diagnosis codes in modelling are inconsistent. AHRQ-Elixhauser is unable to map the vast majority of raw ICD codes. Single-level CCS performs on par with raw codes and is clinically meaningful. Single-level CCS serves as a good baseline as features in clinical prediction tasks. Source code of analyses of MIMIC dataset are available to ensure reproducibility. Abstract: Objective: The primary purpose of this work is to systematically assess the performance trade-offs on clinical prediction tasks of four diagnosis code groupings: AHRQ-Elixhauser, Single-level CCS, truncated ICD-9-CM codes, and raw ICD-9-CM codes. Materials and methods: We used two distinct datasets from different geographic regions and patient populations and train models for three prediction tasks: 1-year mortality following an ICU stay, 30-day mortality following surgery, and 30-day complication following surgery. We run multiple commonly-used binary classification models including penalized logistic regression, random forest, and gradient boosted trees. Model performance is evaluated using the Area Under the Receiver Operating Characteristic (AUROC) and the Area Under the Precision-Recall Curve (AUCPR). Results: Single-level CCS, truncated codes, and raw codes significantly outperformed AHRQ-Elixhauser ICD grouping when predicting 30-day postoperative complication and one-year mortality after ICU admission. The performance across groupings was more similar in the 30-day postoperative mortality prediction task. Discussion: Single-level CCS groupings represent aggregations of raw codes into meaningful clinical concepts and consistently balance interoperability between ICD-9-CM and ICD-10-CM while maintaining strong model performance as measured by AUROC and AUCPR. Key limitations include experimentation across two datasets and three prediction tasks, which although were well labeled and sufficiently prevalent, do not encompass all modeling tasks and outcomes. Conclusion: Single-level CCS groupings may serve as a good baseline for future models that incorporate diagnosis codes as features in clinical prediction tasks. Code and a compute environment summary are provided along with the analyses to enable reproducibility and to support future research.