Creation of a simple natural language processing tool to support an imaging utilization quality dashboard. (May 2017)

• Record Type: Journal Article
• Authors: Swartz, Jordan; Koziatek, Christian; Theobald, Jason; Smith, Silas; Iturrate, Eduardo
• Is Part Of: International journal of medical informatics. Volume 101(2017)
• Page Start: 93
• Page End: 99
• Publication Date: 2017-05
• Subjects: VTE venous thromboembolism -- DVT deep venous thrombosis -- PE pulmonary embolism -- UTZ lower extremity ultrasound -- CTPA computed tomography pulmonary angiogram -- NLP natural language processing -- EHR electronic health record -- CSV comma separated values -- PPV positive predictive value -- NPV negative predictive value -- Sn sensitivity -- Sp specificity -- CI confidence interval; Natural language processing -- Automated text classification -- Pulmonary embolism -- Deep venous thrombosis -- Practice variation -- Benchmarking; Medical informatics -- Periodicals; Information science -- Periodicals; Computers -- Periodicals; Medical technology -- Periodicals; Medical Informatics -- Periodicals; Technology, Medical -- Periodicals; Computers; Information science; Medical informatics; Medical technology; Electronic journals; Periodicals; Electronic journals; 610.285
• DOI: 10.1016/j.ijmedinf.2017.02.011
• Languages: English
• ISSNs: 1386-5056

Abstract

Highlights: An open-source NLP tool was created for those without computer programming experience. The tool was used to classify radiology reports for the presence of thromboembolism. Performance of the tool was excellent and on par with more complex NLP tools. The results of the classification were used to build an imaging quality dashboard. Abstract: Background: Testing for venous thromboembolism (VTE) is associated with cost and risk to patients (e.g. radiation). To assess the appropriateness of imaging utilization at the provider level, it is important to know that provider's diagnostic yield (percentage of tests positive for the diagnostic entity of interest). However, determining diagnostic yield typically requires either time-consuming, manual review of radiology reports or the use of complex and/or proprietary natural language processing software. Objectives: The objectives of this study were twofold: 1) to develop and implement a simple, user-configurable, and open-source natural language processing tool to classify radiology reports with high accuracy and 2) to use the results of the tool to design a provider-specific VTE imaging dashboard, consisting of both utilization rate and diagnostic yield. Methods: Two physicians reviewed a training set of 400 lower extremity ultrasound (UTZ) and computed tomography pulmonary angiogram (CTPA) reports to understand the language used in VTE-positive and VTE-negative reports. The insights from this review informed the arguments to the five modifiable parameters of the NLP tool. A validation set of 2, 000 studies was then independently classified by the reviewers and by the tool; the classifications were compared and the performance of the tool was calculated. Results: The tool was highly accurate in classifying the presence and absence of VTE for both the UTZ (sensitivity 95.7%; 95% CI 91.5–99.8, specificity 100%; 95% CI 100–100) and CTPA reports (sensitivity 97.1%; 95% CI 94.3–99.9, specificity 98.6%; 95% CI 97.8–99.4). The diagnostic yield was then calculated at the individual provider level and the imaging dashboard was created. Conclusions: We have created a novel NLP tool designed for users without a background in computer programming, which has been used to classify venous thromboembolism reports with a high degree of accuracy. The tool is open-source and available for download athttp://iturrate.com/simpleNLP . Results obtained using this tool can be applied to enhance quality by presenting information about utilization and yield to providers via an imaging dashboard.