Improving low-resource machine transliteration by using 3-way transfer learning. (March 2022) - Wu, Chun-Kai; Shih, Chao-Chuang; Wang, Yu-Chun; Tsai, Richard Tzong-Han
Abstract: Transfer learning brings improvement to machine translation by using a resource-rich language pair to pretrain the model and then adapting it to the desired language pair. However, to date, there have been few attempts to tackle machine transliteration with transfer learning. In this article, we propose a method of using source–pivot and pivot–target datasets to improve source–target machine transliteration. Our approach first bridges the source–pivot and pivot–target datasets by reducing the distance between source and pivot embeddings. Then, our model learns to translate from the pivot language to the target language. Finally, the source–target dataset is used to fine tune the model. Our experiments show that our method is superior to the transfer learning method. When implemented with a state-of-the-art source–target translation model from NEWS'18, our transfer learning method can improve the accuracy by 1.1%. Highlights: Unifying the representation of source and pivot language can improve performance. Romanization increases performance of transliteration for non-phonogram language. Parameter sharing is more effective than data sharing. Performance can be increased by adding more auxiliary datasets.