Assessing quality and agreement of structured data in automatic versus manual abstraction of the electronic health record for a clinical epidemiology study. Issue 4 (September 2021)

• Record Type: Journal Article
• Authors: Brazeal, Joseph Grant; Alekseyenko, Alexander V; Li, Hong; Fugal, Mario; Kirchoff, Katie; Marsh, Courtney; Lewin, David N; Wu, Jennifer; Obeid, Jihad; Wallace, Kristin
• Is Part Of: Research methods in medicine & health sciences. Volume 2:Issue 4(2021)
• Page Start: 168
• Page End: 178
• Publication Date: 2021-09
• Subjects: Unstructured data extraction -- automated data extraction -- structured query -- electronic health record characterization -- epidemiology -- investigative techniques -- epidemiologic methods -- data collection -- data accuracy -- negative results -- health information management -- medical records systems -- computerized -- reproducibility of results -- automation -- health services administration -- organization and administration -- meaningful use -- colorectal neoplasms; Medicine -- Research -- Periodicals; Medical sciences -- Research -- Periodicals; 610.7205
• DOI: 10.1177/26320843211061287
• Languages: English
• ISSNs: 2632-0843

Abstract

Objective: We evaluate data agreement between an electronic health record (EHR) sample abstracted by automated characterization with a standard abstracted by manual review. Study Design and Setting: We obtain data for an epidemiology cohort study using standard manual abstraction of the EHR and automated identification of the same patients using a structured algorithm to query the EHR. Summary measures of agreement (e.g., Cohen's kappa) are reported for 12 variables commonly used in epidemiological studies. Results: Best agreement between abstraction methods is observed among demographic characteristics such as age, sex, and race, and for positive history of disease. Poor agreement is found in missing data and negative history, suggesting potential impact for researchers using automated EHR characterization. EHR data quality depends upon providers, who may be influenced by both institutional and federal government documentation guidelines. Conclusion: Automated EHR abstraction discrepancies may decrease power and increase bias; therefore, caution is warranted when selecting variables from EHRs for epidemiological study using an automated characterization approach. Validation of automated methods must also continue to advance in sophistication with other technologies, such as machine learning and natural language processing, to extract non-structured data from the EHR, for application to EHR characterization for clinical epidemiology.