Boosting ICD multi-label classification of health records with contextual embeddings and label-granularity. (May 2020)

• Record Type: Journal Article
• Authors: Blanco, Alberto; Perez-de-Viñaspre, Olatz; Pérez, Alicia; Casillas, Arantza
• Is Part Of: Computer methods and programs in biomedicine. Volume 188(2020)
• Publication Date: 2020-05
• Subjects: Electronic health record -- International classification of diseases -- Multi-label classification -- Recurrent neural networks -- Contextual embeddings -- Label-granularity; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2019.105264
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: The comprehensive documentation of health data is crucial for public health. Task: Automatically coding the diagnostic terms present in a free-text medical record according to the ICD coding system. Challenges: infer Deep Learning models from imbalanced data and capture label-dependencies. Experiments: Five Deep Learning models with exploring several text characterization techniques. Contribution: show the impact of label-granularity on performance and test the contextual embeddings in clinical text mining. Graphical abstract: Abstract: Background and objective: This work deals with clinical text mining, a field of Natural Language Processing applied to biomedical informatics. The aim is to classify Electronic Health Records with respect to the International Classification of Diseases, which is the foundation for the identification of international health statistics, and the standard for reporting diseases and health conditions. Within the framework of data mining, the goal is the multi-label classification, as each health record has assigned multiple International Classification of Diseases codes. We investigate five Deep Learning architectures with a dataset obtained from the Basque Country Health System, and six different perspectives derived from shifts in the input and the output. Methods: We evaluate a Feed Forward Neural Network as the baseline and several Recurrent models based on the Bidirectional GRU architecture, putting our research focus on the text representation layer and testing three variants, from standard word embeddings to meta word embeddings techniques and contextual embeddings. Results: The results showed that the recurrent models overcome the non-recurrent model. The meta word embeddings techniques are capable of beating the standard word embeddings, but the contextual embeddings exhibit as the most robust for the downstream task overall. Additionally, the label-granularity alone has an impact on the classification performance. Conclusions: The contributions of this work are a) a comparison among five classification approaches based on Deep Learning on a Spanish dataset to cope with the multi-label health text classification problem; b) the study of the impact of document length and label-set size and granularity in the multi-label context; and c) the study of measures to mitigate multi-label text classification problems related to label-set size and sparseness.