Variational model for low-resource natural language generation in spoken dialogue systems. (January 2021)

• Record Type: Journal Article
• Authors: Tran, Van-Khanh; Nguyen, Le-Minh
• Is Part Of: Computer speech & language. Volume 65(2021)
• Publication Date: 2021-01
• Subjects: Neural language generation -- Domain adaptation -- Low-resource data -- Variational autoencoder -- Deconvolutional neural network -- CNN -- RNN -- LSTM; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2020.101120
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: We propose a variational-based NLG framework which benefits the generator to quickly adapt to new, unseen domain irrespective of scarce target resources. For domain adaptation, we propose two critics in an adversarial training procedure, which can guide the generator to generate outputs that resemble the sentences drawn from the target domain, which are integrated into a unifying variational domain adaptation architecture that performs acceptably well in a new, unseen domain by using a limited amount data of target domain. For low-resource model designing, we propose a dual latent variable model which benefits the generator to not only outperform the previous methods when there is a sufficient training data, but also perform acceptably well irrespective of scarce in-domain resources. We investigate the effectiveness of the proposed architecture in various scenarios, including domain adaptation, scratch, unsupervised, and semisupervised training with different amount of training dataset. Abstract: Natural Language Generation (NLG) plays a critical role in Spoken Dialogue Systems (SDSs), aims at converting a meaning representation into natural language utterances. Recent deep learning-based generators have shown improving results irrespective of providing sufficient annotated data. Nevertheless, how to build a generator that can effectively utilize as much of knowledge from a low-resource setting data is a crucial issue for NLG in SDSs. This paper presents a variational-based NLG framework to tackle the NLG problem of having limited annotated data in two scenarios, domain adaptation and low-resource in-domain training data. Based on this framework, we propose a novel adversarial domain adaptation NLG taclking the former issue, while the latter issue is also handled by a second proposed dual variational model. We extensively conducted the experiments on four different domains in a variety of training scenarios, in which the experimental results show that the proposed methods not only outperform previous methods when having sufficient training dataset but also show its ability to work acceptably well when there is a small amount of in-domain data or adapt quickly to a new domain with only a low-resource target domain data.