Cross-lingual embeddings with auxiliary topic models. (15th March 2022)

• Record Type: Journal Article
• Authors: Zhou, Dong; Peng, Xiaoya; Li, Lin; Han, Jun-mei
• Is Part Of: Expert systems with applications. Volume 190(2022)
• Publication Date: 2022-03-15
• Subjects: Cross-lingual embeddings -- Topical models -- Word embedding models -- Projection-based methods -- Seed dictionaries; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.116194
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: A novel way to learn effective topical cross-lingual embeddings. Using bilingual topic models to enrich a seed bilingual lexicon. Unsupervised models perform worse than their supervised competitors. Abstract: Projection-based methods for generating high-quality Cross-Lingual Embeddings (CLEs) have shown state-of-the-art performance in many multilingual applications. Supervised methods that rely on character-level information or unsupervised methods that need only monolingual information are both popular and have their pros and cons. However, there are still problems in terms of the quality of monolingual word embedding spaces and the generation of the seed dictionaries. In this work, we aim to generate effective CLEs with auxiliary Topic Models. We utilize both monolingual and bilingual topic models in the procedure of generating monolingual embedding spaces and seed dictionaries for projection. We present a comprehensive evaluation of our proposed model through the means of bilingual lexicon extraction, cross-lingual semantic word similarity and cross-lingual document classification tasks. We show that our proposed model outperforms existing supervised and unsupervised CLE models built on basic monolingual embedding spaces and seed dictionaries. It also exceeds CLE models generated from representative monolingual topical word embeddings.