Low-resource text classification using domain-adversarial learning. (July 2020)

• Record Type: Journal Article
• Authors: Grießhaber, Daniel; Vu, Ngoc Thang; Maucher, Johannes
• Is Part Of: Computer speech & language. Volume 62(2020)
• Publication Date: 2020-07
• Subjects: NLP -- Low-resource -- Deep learning -- Domain-adversarial; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2019.101056
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Domain-Adversarial training works for complex NLP model architectures in low- and no- resource settings. Pre-trained multilingual word-vectors are not necessary when training such models as they can be learned ad-hoc. Multi-Domain training in beneficial if training data is available. We explore and analyse the produced features of the trained models. Abstract: Deep learning techniques have recently shown to be successful in many natural language processing tasks forming state-of-the-art systems. They require, however, a large amount of annotated data which is often missing. This paper explores the use of domain-adversarial learning as a regularizer to avoid overfitting when training domain invariant features for deep, complex neural networks in low-resource and zero-resource settings in new target domains or languages. In case of new languages, we show that monolingual word vectors can be directly used for training without prealignment. Their projection into a common space can be learnt ad-hoc at training time reaching the final performance of pretrained multilingual word vectors.