MulTed: a multilingual aligned and tagged parallel corpus. Issue 1 (17th July 2020)

• Record Type: Journal Article
• Authors: Zeroual, Imad; Lakhouaja, Abdelhak
• Is Part Of: Applied computing and informatics. Volume 18:Issue 1/2(2022)
• Page Start: 61
• Page End: 73
• Publication Date: 2020-07-17
• Subjects: Multilingual parallel corpora -- Natural Language Processing -- Linguistic resources -- Sentence alignment -- Treetagger; Information science -- Periodicals; Information storage and retrieval systems -- Periodicals; 004
• DOI: 10.1016/j.aci.2018.12.003
• Languages: English
• ISSNs: 2210-8327

Abstract

Abstract : Recently, more data-driven approaches are demanding multilingual parallel resources primarily in the cross-language studies. To meet these demands, building multilingual parallel corpora are becoming the focus of many Natural Language Processing (NLP) scientific groups. Unlike monolingual corpora, the number of available multilingual parallel corpora is limited. In this paper, the MulTed, a corpus of subtitles extracted from TEDx talks is introduced. It is multilingual, Part of Speech (PoS) tagged, and bilingually sentence-aligned with English as a pivot language. This corpus is designed for many NLP applications, where the sentence-alignment, the PoS tagging, and the size of corpora are influential such as statistical machine translation, language recognition, and bilingual dictionary generation. Currently, the corpus has subtitles that cover 1100 talks available in over 100 languages. The subtitles are classified based on a variety of topics such as Business, Education, and Sport. Regarding the PoS tagging, the Treetagger, a language-independent PoS tagger, is used; then, to make the PoS tagging maximally useful, a mapping process to a universal common tagset is performed. Finally, we believe that making the MulTed corpus available for a public use can be a significant contribution to the literature of NLP and corpus linguistics, especially for under-resourced languages.