Character expression for spoken dialogue systems with semi-supervised learning using Variational Auto-Encoder. (April 2023)

• Record Type: Journal Article
• Authors: Yamamoto, Kenta; Inoue, Koji; Kawahara, Tatsuya
• Is Part Of: Computer speech & language. Volume 79(2023)
• Publication Date: 2023-04
• Subjects: Spoken dialogue system -- Character -- Semi-supervised learning -- Variational auto-encoder (VAE); Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2022.101469
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Character of spoken dialogue systems is important not only for giving a positive impression of the system but also for gaining rapport from users. We have proposed a character expression model for spoken dialogue systems. The model expresses three character traits (extroversion, emotional instability, and politeness) of spoken dialogue systems by controlling spoken dialogue behaviors: utterance amount, backchannel, filler, and switching pause length. One major problem in training this model is that it is costly and time-consuming to collect many pair data of character traits and behaviors. To address this problem, semi-supervised learning is proposed based on a variational auto-encoder that exploits both the limited amount of labeled pair data and unlabeled corpus data. It was confirmed that the proposed model can express given characters more accurately than a baseline model with only supervised learning. We also implemented the character expression model in a spoken dialogue system for an autonomous android robot, and then conducted a subjective experiment with 75 university students to confirm the effectiveness of the character expression for specific dialogue scenarios. The results showed that expressing a character in accordance with the dialogue task by the proposed model improves the user's impression of the appropriateness in formal dialogue such as job interview.