Multi-way, multilingual neural machine translation. (September 2017)

• Record Type: Journal Article
• Authors: Firat, Orhan; Cho, Kyunghyun; Sankaran, Baskaran; Yarman Vural, Fatos T.; Bengio, Yoshua
• Is Part Of: Computer speech & language. Volume 45(2017)
• Page Start: 236
• Page End: 252
• Publication Date: 2017-09
• Subjects: Neural machine translation -- Multi-lingual -- Low resource translation; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2016.10.006
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: The first attention-based neural-MT for multi-way, multilingual translation is proposed. Multi-way multilingual model is tested on more than 8 languages (En, Fr, Cz, De, Ru, Fi, Tr and Uz). It achieves the translation quality comparable to single-pair NMT's with less parameters. Single attention mechanism supports to align between multiple pairs and directions. Outperforms conventional SMT system on low-resource translation tasks. Abstract: We propose multi-way, multilingual neural machine translation. The proposed approach enables a single neural translation model to translate between multiple languages, with a number of parameters that grows only linearly with the number of languages. This is made possible by having a single attention mechanism that is shared across all language pairs. We train the proposed multi-way, multilingual model on ten language pairs from WMT'15 simultaneously and observe clear performance improvements over models trained on only one language pair. We empirically evaluate the proposed model on low-resource language translation tasks. In particular, we observe that the proposed multilingual model outperforms strong conventional statistical machine translation systems on Turkish-English and Uzbek-English by incorporating the resources of other language pairs.