THU0612 Development and Validation of An Accurate and Precise Natural Language Processing System To Capture Rheumatoid Arthritis Disease Activity Measures. (15th July 2016)

• Record Type: Journal Article
• Authors: Cannon, G.W.; Mehortra, S.; South, B.R.; Leng, J.; Lu, C.-C.; Sauer, B.C.
• Is Part Of: Annals of the rheumatic diseases. Volume 75(2016)Supplement 2
• Page Start: 414
• Page End: 414
• Publication Date: 2016-07-15
• Subjects: Rheumatism -- Periodicals; 616.723005
• DOI: 10.1136/annrheumdis-2016-eular.1890
• Languages: English
• ISSNs: 0003-4967

Abstract

Abstract : Background: Systems to electronically retrieve disease activity measures in medical records would significantly aid rheumatoid arthritis (RA) management and enhance epidemiologic research. As part of the national VA rheumatoid arthritis (VARA) registry our clinic measures 28 joint disease activity score (DAS28) by manual extraction (ManEx). Objectives: 1. To develop a natural language processing (NLP) system to extract core measures from patient clinic note text and erythrocyte sedimentation rate (ESR) from the laboratory data to calculate DAS28. 2. To compare the accuracy and precision of the NLP DAS28 with ManEx DAS28. 3. To validate these findings in a second RA cohort. Methods: An NLP application used clinical notes between January 1, 2015 and September 30, 2015 for RA patients in VARA (n=207) at our single site to extract tender joint count (TJC), swollen joint count (SJC), and patient global assessment (PtGA) from note text. The ESR value within 2 weeks closest to the clinic visit was combined with TJC, SJC, and PtGA to calculate DAS28. During the same observation period, ManEx of DAS28 observations were also identified. Discrepancies between ManEx and NLP were resolved by investigator review of the clinic notes. An analysis of all notes identified either by ManEx or NLP were evaluated for accuracy (% of values correctly detected in all notes) and precision (% of retrieved values by either ManEx or NLP correctly identified by that specific method). The NLP system was validated in a second cohort of RA patients in VARA (n=214) at our site between April 1, 2014 to December 31, 2014. Results: The derivation set contained a total of 472 notes; 440 (93%) identified by both ManEx and NLP; 20 (4%) by ManEx alone and 12 (3%) notes by NLP alone. The accuracy and precision for TJC, SJC, PtGA, ESR and DAS28 are as noted in the table. Failure of NLP to detect core measures was because of use of incorrect note template (n=5, 1.0%) and template modification (n=18, 3.8%); however, when detected NLP precision was near 100%. Failure of ManEx was a result of missed notes (n=19, 4.0%) and data entry errors (n=13, 2.8%). Similar results were seen in the validations set of notes (n=484). Conclusions: This accurate and precise NLP system can retrieve DAS28 from clinic notes to aid in clinical care and research activities. This method can be implemented at other VARA sites and other research networks to collect RA outcomes. Disclosure of Interest: None declared