Tax2vec: Constructing Interpretable Features from Taxonomies for Short Text Classification. (January 2021)

• Record Type: Journal Article
• Authors: Škrlj, Blaž; Martinc, Matej; Kralj, Jan; Lavrač, Nada; Pollak, Senja
• Is Part Of: Computer speech & language. Volume 65(2021)
• Publication Date: 2021-01
• Subjects: taxonomies -- vectorization -- text classification -- short documents -- feature construction -- semantic enrichment; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2020.101104
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: The use of background knowledge is largely unexploited in text classification tasks. This paper explores word taxonomies as means for constructing new semantic features, which may improve the performance and robustness of the learned classifiers. We propose tax2vec, a parallel algorithm for constructing taxonomy-based features, and demonstrate its use on six short text classification problems: prediction of gender, personality type, age, news topics, drug side effects and drug effectiveness. The constructed semantic features, in combination with fast linear classifiers, tested against strong baselines such as hierarchical attention neural networks, achieves comparable classification results on short text documents. The algorithm's performance is also tested in a few-shot learning setting, indicating that the inclusion of semantic features can improve the performance in data-scarce situations. The tax2vec capability to extract corpus-specific semantic keywords is also demonstrated. Finally, we investigate the semantic space of potential features, where we observe a similarity with the well known Zipf's law.