Using Text Mining to Track Outbreak Trends in Global Surveillance of Emerging Diseases: ProMED-mail. (1st July 2021)

• Record Type: Journal Article
• Authors: You, Jingxian; Expert, Paul; Costelloe, Céire
• Is Part Of: Journal of the Royal Statistical Society. Volume 184:Number 4(2021)
• Page Start: 1245
• Page End: 1259
• Publication Date: 2021-07-01
• Subjects: communicable disease -- data mining -- emerging; Social sciences -- Statistical methods -- Periodicals; Statistics -- Periodicals; 300.15195
• DOI: 10.1111/rssa.12721
• Languages: English
• ISSNs: 0964-1998

Abstract

Abstract: ProMED-mail (Program for Monitoring Emerging Disease) is an international disease outbreak monitoring and early warning system. Every year, users contribute thousands of reports that include reference to infectious diseases and toxins. However, due to the uneven distribution of the reports for each disease, traditional statistics-based text mining techniques, represented by term frequency-related algorithm, are not suitable. Thus, we conducted a study in three steps (i) report filtering, (ii) keyword extraction from reports and finally (iii) word co-occurrence network analysis to fill the gap between ProMED and its utilization. The keyword extraction was performed with the TextRank algorithm, keywords co-occurrence networks were then produced using the top keywords from each document and multiple network centrality measures were computed to analyse the co-occurrence networks. We used two major outbreaks in recent years, Ebola, 2014 and Zika 2015, as cases to illustrate and validate the process. We found that the extracted information structures are consistent with World Health Organisation description of the timeline and phases of the epidemics. Our research presents a pipeline that can extract and organize the information to characterize the evolution of epidemic outbreaks. It also highlights the potential for ProMED to be utilized in monitoring, evaluating and improving responses to outbreaks.