Development of a natural language processing algorithm to extract seizure types and frequencies from the electronic health record. (October 2022)

• Record Type: Journal Article
• Authors: Decker, Barbara M.; Turco, Alexandra; Xu, Jian; Terman, Samuel W.; Kosaraju, Nikitha; Jamil, Alisha; Davis, Kathryn A.; Litt, Brian; Ellis, Colin A.; Khankhanian, Pouya; Hill, Chloe E.
• Is Part Of: Seizure. Volume 101(2022)
• Page Start: 48
• Page End: 51
• Publication Date: 2022-10
• Subjects: Epilepsy -- Natural language processing -- Seizure frequency -- Electronic health record -- Automated extraction; Epilepsy -- Periodicals; Epilepsy -- Periodicals; Seizures -- Periodicals; Épilepsie -- Périodiques; Electronic journals; Electronic journals; 616.853
• DOI: 10.1016/j.seizure.2022.07.010
• Languages: English
• ISSNs: 1059-1311

Abstract

Highlights: Seizure frequency measurement is an important epilepsy quality metric. Natural Language Processing (NLP) mines unstructured text from clinical encounters. This NLP algorithm extracts seizure frequencies for multiple seizure types. Algorithm performance was acceptable internally but had poor generalizability. NLP extraction is feasible but large-scale implementation is challenging. Abstract: Objective: To develop a natural language processing (NLP) algorithm to abstract seizure types and frequencies from electronic health records (EHR). Background: Seizure frequency measurement is an epilepsy quality metric. Yet, abstraction of seizure frequency from the EHR is laborious. We present an NLP algorithm to extract seizure data from unstructured text of clinic notes. Algorithm performance was assessed at two epilepsy centers. Methods: We developed a rules-based NLP algorithm to recognize terms related to seizures and frequency within the text of an outpatient encounter. Algorithm output (e.g. number of seizures of a particular type within a time interval) was compared to seizure data manually annotated by two expert reviewers ("gold standard"). The algorithm was developed from 150 clinic notes from institution #1 (development set), then tested on a separate set of 219 notes from institution #1 (internal test set) with 248 unique seizure frequency elements. The algorithm was separately applied to 100 notes from institution #2 (external test set) with 124 unique seizure frequency elements. Algorithm performance was measured by recall (sensitivity), precision (positive predictive value), and F1 score (geometric mean of precision and recall). Results: In the internal test set, the algorithm demonstrated 70% recall (173/248), 95% precision (173/182), and 0.82 F1 score compared to manual review. Algorithm performance in the external test set was lower with 22% recall (27/124), 73% precision (27/37), and 0.40 F1 score. Conclusions: These results suggest NLP extraction of seizure types and frequencies is feasible, though not without challenges in generalizability for large-scale implementation.