Simple methods to overcome the limitations of general word representations in natural language processing tasks. (January 2020)

• Record Type: Journal Article
• Authors: Yu, Hongyeon; An, Jaehyun; Yoon, Jeongmin; Kim, Hyemin; Ko, Youngjoong
• Is Part Of: Computer speech & language. Volume 59(2020)
• Page Start: 91
• Page End: 113
• Publication Date: 2020-01
• Subjects: General word representation -- Task-specific word representation -- Out-of-vocabulary problem -- Natural language processing; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2019.04.009
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: We propose a simple method to obtain task-specific word representation. We propose to handle out the OOV problem by subword and mapping approaches. The proposed methods achieved performance improvement in all four Korean tasks. Abstract: Although general word representations (GWRs) by skip-gram or GloVe have been widely used in many natural language processing (NLP) tasks with considerable success, they require further improvement. First, a GWR only represents general information of a word, even though task-oriented information can be more useful in specific tasks. Second, a GWR cannot avoid the out-of-vocabulary (OOV) problem. Thus, some recent studies have proposed methods based on an additional complex model or deep knowledge of resources for each specific task. Although such methods have the potential for improved performance, we believe that the baseline systems of each NLP task are already expensive; hence, making them more complex would be problematic for real-world applications. Therefore, the objective of this study is to overcome the limitations of GWRs by developing simple but effective methods for task-specific word representations (TSWRs) and OOV representations (OOVRs). The proposed methods achieved state-of-the-art performance in four Korean NLP tasks, namely part-of-speech tagging, named entity recognition, dependency parsing, and semantic role labeling.