Improving low-resource machine transliteration by using 3-way transfer learning. (March 2022)

• Record Type: Journal Article
• Authors: Wu, Chun-Kai; Shih, Chao-Chuang; Wang, Yu-Chun; Tsai, Richard Tzong-Han
• Is Part Of: Computer speech & language. Volume 72(2022)
• Publication Date: 2022-03
• Subjects: Machine transliteration -- Transfer learning -- Low resource; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2021.101283
• Languages: English
• ISSNs: 0885-2308

Abstract

Abstract: Transfer learning brings improvement to machine translation by using a resource-rich language pair to pretrain the model and then adapting it to the desired language pair. However, to date, there have been few attempts to tackle machine transliteration with transfer learning. In this article, we propose a method of using source–pivot and pivot–target datasets to improve source–target machine transliteration. Our approach first bridges the source–pivot and pivot–target datasets by reducing the distance between source and pivot embeddings. Then, our model learns to translate from the pivot language to the target language. Finally, the source–target dataset is used to fine tune the model. Our experiments show that our method is superior to the transfer learning method. When implemented with a state-of-the-art source–target translation model from NEWS'18, our transfer learning method can improve the accuracy by 1.1%. Highlights: Unifying the representation of source and pivot language can improve performance. Romanization increases performance of transliteration for non-phonogram language. Parameter sharing is more effective than data sharing. Performance can be increased by adding more auxiliary datasets.