Traditional Chinese medicine clinical records classification with BERT and domain specific corpora. (24th September 2019)

• Record Type: Journal Article
• Authors: Yao, Liang; Jin, Zhe; Mao, Chengsheng; Zhang, Yin; Luo, Yuan
• Is Part Of: Journal of the American Medical Informatics Association. Volume 26:Number 12(2019)
• Page Start: 1632
• Page End: 1636
• Publication Date: 2019-09-24
• Subjects: natural language processing -- clinical records classification -- BERT -- domain knowledge -- TCM; Medical informatics -- Periodicals; Information Services -- Periodicals; Medical Informatics -- Periodicals; Médecine -- Informatique -- Périodiques; Informatica; Geneeskunde; Informatique médicale; Computer network resources; Electronic journals; 610.285
• DOI: 10.1093/jamia/ocz164
• Languages: English
• ISSNs: 1067-5027

Abstract

Abstract: Traditional Chinese Medicine (TCM) has been developed for several thousand years and plays a significant role in health care for Chinese people. This paper studies the problem of classifying TCM clinical records into 5 main disease categories in TCM. We explored a number of state-of-the-art deep learning models and found that the recent Bidirectional Encoder Representations from Transformers can achieve better results than other deep learning models and other state-of-the-art methods. We further utilized an unlabeled clinical corpus to fine-tune the BERT language model before training the text classifier. The method only uses Chinese characters in clinical text as input without preprocessing or feature engineering. We evaluated deep learning models and traditional text classifiers on a benchmark data set. Our method achieves a state-of-the-art accuracy 89.39% ± 0.35%, Macro F1 score 88.64% ± 0.40% and Micro F1 score 89.39% ± 0.35%. We also visualized attention weights in our method, which can reveal indicative characters in clinical text.