Validation of an internationally derived patient severity phenotype to support COVID-19 analytics from electronic health record data. (4th May 2021)

• Record Type: Journal Article
• Authors: Klann, Jeffrey G; Estiri, Hossein; Weber, Griffin M; Moal, Bertrand; Avillach, Paul; Hong, Chuan; Tan, Amelia L M; Beaulieu-Jones, Brett K; Castro, Victor; Maulhardt, Thomas; Geva, Alon; Malovini, Alberto; South, Andrew M; Visweswaran, Shyam; Morris, Michele; Samayamuthu, Malarkodi J; Omenn, Gilbert S; Ngiam, Kee Yuan; Mandl, Kenneth D; Boeker, Martin; Olson, Karen L; Mowery, Danielle L; Follett, Robert W; Hanauer, David A; Bellazzi, Riccardo; Moore, Jason H; Loh, Ne-Hooi Will; Bell, Douglas S; Wagholikar, Kavishwar B; Chiovato, Luca; Tibollo, Valentina; Rieg, Siegbert; Li, Anthony L L J; Jouhet, Vianney; Schriver, Emily; Xia, Zongqi; Hutch, Meghan; Luo, Yuan; Kohane, Isaac S; Brat, Gabriel A; Murphy, Shawn N;
• Is Part Of: Journal of the American Medical Informatics Association. Volume 28:Number 7(2021)
• Page Start: 1411
• Page End: 1420
• Publication Date: 2021-05-04
• Subjects: novel coronavirus -- disease severity -- computable phenotype -- medical informatics -- data networking -- data interoperability; Medical informatics -- Periodicals; Information Services -- Periodicals; Medical Informatics -- Periodicals; Médecine -- Informatique -- Périodiques; Informatica; Geneeskunde; Informatique médicale; Computer network resources; Electronic journals; 610.285
• DOI: 10.1093/jamia/ocab018
• Languages: English
• ISSNs: 1067-5027

Abstract

Abstract: Objective: The Consortium for Clinical Characterization of COVID-19 by EHR (4CE) is an international collaboration addressing coronavirus disease 2019 (COVID-19) with federated analyses of electronic health record (EHR) data. We sought to develop and validate a computable phenotype for COVID-19 severity. Materials and Methods: Twelve 4CE sites participated. First, we developed an EHR-based severity phenotype consisting of 6 code classes, and we validated it on patient hospitalization data from the 12 4CE clinical sites against the outcomes of intensive care unit (ICU) admission and/or death. We also piloted an alternative machine learning approach and compared selected predictors of severity with the 4CE phenotype at 1 site. Results: The full 4CE severity phenotype had pooled sensitivity of 0.73 and specificity 0.83 for the combined outcome of ICU admission and/or death. The sensitivity of individual code categories for acuity had high variability—up to 0.65 across sites. At one pilot site, the expert-derived phenotype had mean area under the curve of 0.903 (95% confidence interval, 0.886-0.921), compared with an area under the curve of 0.956 (95% confidence interval, 0.952-0.959) for the machine learning approach. Billing codes were poor proxies of ICU admission, with as low as 49% precision and recall compared with chart review. Discussion: We developed a severity phenotype using 6 code classes that proved resilient to coding variability across international institutions. In contrast, machine learning approaches may overfit hospital-specific orders. Manual chart review revealed discrepancies even in the gold-standard outcomes, possibly owing to heterogeneous pandemic conditions. Conclusions: We developed an EHR-based severity phenotype for COVID-19 in hospitalized patients and validated it at 12 international sites.