HSCJN: A holistic semantic constraint joint network for diverse response generation. (January 2021)

• Record Type: Journal Article
• Authors: Wang, Yiru; Si, Pengda; Lei, Zeyang; Xun, Guangxu; Yang, Yujiu
• Is Part Of: Computer speech & language. Volume 65(2021)
• Publication Date: 2021-01
• Subjects: Dialogue generation -- Diversity -- Holistic semantics -- Word prediction -- Entropy regularizer; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2020.101135
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: A novel diversity-promoting joint training network for open-domain dialogue generation. We introduce future information during the decoding stage, thus the generation of each word can leverage complete information of the target utterance. We devise a maximum entropy regularizer to alleviate the over-estimation of high-frequency words. Our network introduces more linguistic information from target utterances to increase diversity, and captures direct semantic information to better constrain relevance simultaneously. The method can be applied to any sequence-to-sequence architecture. Abstract: The sequence-to-sequence (Seq2Seq) model generates target words iteratively given the previously observed words during the decoding process, which results in the loss of the holistic semantics in the target response and the complete semantic relationship between responses and dialogue histories. In this paper, we propose a generic diversity-promoting joint network, called Holistic Semantic Constraint Joint Network (HSCJN), enhancing the global sentence information, and then regularizing the objective function with penalizing the low entropy output during the training stage. Our network introduces more target information to improve diversity and captures direct semantic information to better constrain relevance simultaneously. Moreover, the proposed method can be easily applied to any Seq2Seq structure. Extensive experiments on several dialogue corpuses show that our method effectively improves both semantic consistency and diversity of generated responses, and achieves better performance than other competitive methods.