Sequential neural networks for noetic end-to-end response selection. (July 2020)

• Record Type: Journal Article
• Authors: Chen, Qian; Wang, Wen
• Is Part Of: Computer speech & language. Volume 62(2020)
• Publication Date: 2020-07
• Subjects: DSTC7 -- Response selection -- ESIM -- BERT -- End-to-end -- Sequential matching approaches; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2020.101072
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: An ESIM model based system ranked top 1 on both datasets of DSTC7 noetic end-to-end response selection track. An accurate and efficient two-step approach for response selection from a large amount of candidates. Heuristic data augmentation and optimized sampling approaches improved the ESIM model performance. Training task-specific word embeddings incorporating external domain knowledge for the ESIM model. An effective and efficient approach for ensembling models trained with different parameter initializations and structures. Extensive ablation analyses for various factors contributing to the ESIM model performance. Systematic comparisons between ESIM and BERT models for response selection and ablation analyses for the BERT model. Abstract: The noetic end-to-end response selection challenge as one track in the 7th Dialog System Technology Challenges (DSTC7) aims to push the state of the art of utterance classification for real world goal-oriented dialog systems, for which participants need to select the correct next utterances from a set of candidates for the multi-turn context. This paper presents our systems that are ranked top 1 on both datasets under this challenge, one focused and small (Advising) and the other more diverse and large (Ubuntu). Previous state-of-the-art models use hierarchy-based (utterance-level and token-level) neural networks to explicitly model the interactions among different turns' utterances for context modeling. In this paper, we investigate a sequential matching model based only on chain sequence for multi-turn response selection. Our results demonstrate that the potentials of sequential matching approaches have not yet been fully exploited in the past for multi-turn response selection. In addition to ranking top 1 in the challenge, the proposed model outperforms all previous models, including state-of-the-art hierarchy-based models, on two large-scale public multi-turn response selection benchmark datasets.