Building an annotated corpus for the Albanian language using bilingual projections and regular expressions. Issue 2 (8th July 2019)

• Record Type: Journal Article
• Authors: Kadriu, Arbana
• Is Part Of: International journal of knowledge engineering and data mining. Volume 6:Issue 2(2019)
• Page Start: 105
• Page End: 121
• Publication Date: 2019-07-08
• Subjects: POS tagging -- Albanian language -- bilingual projection -- corpus creation; Knowledge representation (Information theory) -- Periodicals; Data mining -- Periodicals; 006.305
• Languages: English
• ISSNs: 1755-2087

Abstract

We present research done on creating an annotated corpus for the Albanian language. This corpus is achieved combining unsupervised part-of-speech tagging using bilingual projections with regular expressions. Albanian-English text from a free parallel corpus for the Balkan languages is used as a basis. The annotating process is based on the universal part-of-speech tag system. As the result of the projected tagging, we gained a tagged corpus in Albanian for 60, 000 sentences. We investigate the main pitfalls in the output gained from the parallel projection and use this analysis to define replacement rules for part of our tagged corpus, which will change 18% of the initial text. We investigate the effectiveness of the tagged corpus using four different part-of-speech taggers, the best result of which is of 94% accuracy. We discuss further improvements to this corpus, which to our knowledge is the biggest annotated corpus in Albanian.