Zero-shot cross-lingual transfer language selection using linguistic similarity. Issue 3 (May 2023)

• Record Type: Journal Article
• Authors: Eronen, Juuso; Ptaszynski, Michal; Masui, Fumito
• Is Part Of: Information processing & management. Volume 60:Issue 3(2023)
• Publication Date: 2023-05
• Subjects: Multilingual natural language processing -- Zero-shot learning -- Transfer learning -- Linguistics -- Language similarity; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2022.103250
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: We study the selection of transfer languages for different Natural Language Processing tasks, specifically sentiment analysis, named entity recognition and dependency parsing. In order to select an optimal transfer language, we propose to utilize different linguistic similarity metrics to measure the distance between languages and make the choice of transfer language based on this information instead of relying on intuition. We demonstrate that linguistic similarity correlates with cross-lingual transfer performance for all of the proposed tasks. We also show that there is a statistically significant difference in choosing the optimal language as the transfer source instead of English. This allows us to select a more suitable transfer language which can be used to better leverage knowledge from high-resource languages in order to improve the performance of language applications lacking data. For the study, we used datasets from eight different languages from three language families. Highlights: Zero-shot cross-lingual transfer has potential in achieving results close to that of monolingual settings. A suitable transfer language can be found by using Linguistic similarity for many NLP tasks. Selecting a transfer language based on intuition or simply by defaulting to English often results in performance loss. The linguistic similarity metric quantified from World Atlas of Language Structures is comparably robust.