Adversarial subsequences for unconditional text generation. (November 2021)

• Record Type: Journal Article
• Authors: Chen, Xingyuan; Jin, Peng; Li, Yanzhe; Zhang, Jiuhua; Dai, Xinyu; Chen, Jiajun
• Is Part Of: Computer speech & language. Volume 70(2021)
• Publication Date: 2021-11
• Subjects: Unconditional text generation -- GAN -- Subsequences; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2021.101242
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: Apply adversarial learning to subsequences, rather than only to the entire sequences, to improve unconditional text generation. Implement this mechanism in GAN-based methods, including both reinforcement learning-based and reinforcement learning-free. On two benchmark datasets, our method outperforms the state-of-the-art model in both quality and diversity simultaneously. Abstract: Generative Adversarial Nets (GAN) has been successfully introduced to unconditional generating text to alleviate exposure bias. However, the discriminator in this model only evaluates the entire sequence, which causes feedback sparsity and mode collapse. To tackle these problems, we propose a novel mechanism. The mechanism first segments the entire sequence into several subsequences. Then, these subsequences, together with the entire sequence, are evaluated individually by the discriminator. Finally, these feedback signals are all used to guide the learning of GAN. This mechanism learns the generation of both the entire sequence and the subsequences simultaneously. Learning to generate subsequences is easy and is helpful in generating an entire sequence. It is easy to improve the existing GAN-based models with this mechanism. Although Li et al. (2017) segments the generated responses in a conditional text generation task, i.e., a dialogue system, they conclude it is weaker than the Monte Carlo search. However, for unconditional text generation, we observe that adversarial learning on subsequences works well. We rebuild three previous models with our mechanism, and the experimental results on two benchmark datasets show these models are improved greatly and outperform the state-of-the-art model.