A Bi-LSTM memory network for end-to-end goal-oriented dialog learning. (January 2019)

• Record Type: Journal Article
• Authors: Kim, Byoungjae; Chung, KyungTae; Lee, Jeongpil; Seo, Jungyun; Koo, Myoung-Wan
• Is Part Of: Computer speech & language. Volume 53(2019)
• Page Start: 217
• Page End: 230
• Publication Date: 2019-01
• Subjects: Bi-LSTM memory network -- End-to-end goal-oriented dialog -- Human–computer interaction; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2018.06.005
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: An end-to-end goal-oriented dialog learning system is proposed. Learning is conducted using a Bi-LSTM memory network model. Performance increased with the use of metadata. The Bi-LSTM memory network improves on the original and dynamic memory networks. Abstract: We develop a model to satisfy the requirements of Dialog System Technology Challenge 6 (DSTC6) Track 1: building an end-to-end dialog systems for goal-oriented applications. This task involves learning a dialog policy from transactional dialogs in a given domain. Automatic system responses are generated using given task-oriented dialog data (http://workshop.colips.org/dstc6/index.html ). As this task has a similar structure to a question answering task (Weston et al., 2015), we employ the MemN2N architecture (Sukhbaatar et al., 2015), which outperforms models based on recurrent neural networks or long short-term memory (LSTM). However, two problems arise when applying this model to the DSTC6 task. First, we encounter an out-of-vocabulary problem, which we resolve by categorizing the metadata types of words that exist in the knowledge base; the metadata is similar to the named entity. Second, the original memory network model has a weak ability to reflect sufficient temporal information, because it only uses sentence-level embeddings. Therefore, we add bidirectional LSTM (Bi-LSTM) at the beginning of the model to better reflect temporal information. The experimental results demonstrate that our model reflects temporal features well. Furthermore, our model achieves state-of-the-art performance among the memory networks, and is comparable to hybrid code networks (Ham et al., 2017) and hierarchical LSTM model (Bai et al., 2017) which is not an end-to-end architecture.