Searching for discriminative words in multidimensional continuous feature space. (January 2019)

• Record Type: Journal Article
• Authors: Sajgalik, Marius; Barla, Michal; Bielikova, Maria
• Is Part Of: Computer speech & language. Volume 53(2019)
• Page Start: 276
• Page End: 301
• Publication Date: 2019-01
• Subjects: Text categorisation -- Distributed representation -- Feature vectors -- Word vectors -- NLP; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2017.10.002
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: A novel method for extracting relevant and discriminative keywords is proposed. Effective use of latent feature vector space for keyword extraction. Discriminative metrics can be used to boost discriminativeness of keywords. Using just 10 keywords per document, we can achieve state of the art results in text categorisation. Abstract: Word feature vectors have been proven to improve many natural language processing tasks. With recent advances in unsupervised learning of these feature vectors, it became possible to train it with much more data, which also resulted in better quality of learned features. Since it learns joint probability of latent features of words, it has the advantage that we can train it without any prior knowledge about the goal task we want to solve. We aim to evaluate the universal applicability property of feature vectors, which has been already proven to hold for many standard NLP tasks like part-of-speech tagging or syntactic parsing. In our case, we want to understand the topical focus of text documents and design an efficient representation suitable for discriminating different topics. The discriminativeness can be evaluated adequately on text categorisation task. We propose a novel method to extract discriminative keywords from documents. We utilise word feature vectors to understand the relations between words better and also understand the latent topics which are discussed in the text and not mentioned directly but inferred logically. We also present a simple way to calculate document feature vectors out of extracted discriminative words. We evaluate our method on the four most popular datasets for text categorisation. We show how different discriminative metrics influence the overall results. We demonstrate the effectiveness of our approach by achieving state-of-the-art results on text categorisation task using just a small number of extracted keywords. We prove that word feature vectors can substantially improve the topical inference of documents' meaning. We conclude that distributed representation of words can be used to build higher levels of abstraction as we demonstrate and build feature vectors of documents. Our method can help in any multi-domain environment to automatically extract discriminative keywords. It can be used to organise and search documents more efficiently.