Feature engineering and machine learning for causality assessment in pharmacovigilance: Lessons learned from application to the FDA Adverse Event Reporting System. (August 2021)

• Record Type: Journal Article
• Authors: Kreimeyer, Kory; Dang, Oanh; Spiker, Jonathan; Muñoz, Monica A.; Rosner, Gary; Ball, Robert; Botsis, Taxiarchis
• Is Part Of: Computers in biology and medicine. Volume 135(2021)
• Publication Date: 2021-08
• Subjects: Pharmacovigilance -- Case classification -- Clinical natural language processing -- Decision support -- Causality assessment; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2021.104517
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Background: Our objective was to support the automated classification of Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS) reports for their usefulness in assessing the possibility of a causal relationship between a drug product and an adverse event. Method: We used a data set of 326 redacted FAERS reports that was previously annotated using a modified version of the World Health Organization–Uppsala Monitoring Centre criteria for drug causality assessment by a group of SEs at the FDA and supported a similar study on the classification of reports using supervised machine learning and text engineering methods. We explored many potential features, including the incorporation of natural language processing on report text and information from external data sources, for supervised learning and developed models for predicting the classification status of reports. We then evaluated the models on a larger data set of previously unseen reports. Results: The best-performing models achieved recall and F1 scores on both data sets above 0.80 for the identification of assessable reports (i.e. those containing enough information to make an informed causality assessment) and above 0.75 for the identification of reports meeting at least a Possible causality threshold. Conclusions: Causal inference from FAERS reports depends on many components with complex logical relationships that are yet to be made fully computable. Efforts focused on readily addressable tasks, such as quickly eliminating unassessable reports, fit naturally in SE's thought processes to provide real enhancements for FDA workflows. Highlights: Assessing the increasing number of FAERS reports benefits from the use of automated approaches Case classification for causality depends on information internal and external to the report Text engineering features alone cannot support case classification for causality Clinical and temporal features are critical information in FAERS narratives We demonstrated high efficiency in classifying reports for having inadequate information for assessment