Phone-based identification of language in code-mixed social network data. Issue 4 (4th July 2017)

• Record Type: Journal Article
• Authors: Saharia, Navanath
• Is Part Of: Journal of statistics & management systems. Volume 20:Issue 4(2017)
• Page Start: 565
• Page End: 574
• Publication Date: 2017-07-04
• Subjects: N-gram -- code-mixed data -- language identification -- Assamese-English; 68T50; Statistics -- Periodicals; Mathematical models -- Periodicals; Mathematical models; Statistics; Periodicals; 519.5
• DOI: 10.1080/09720510.2017.1395176
• Languages: English
• ISSNs: 0972-0510

Abstract

Abstract: We explore frequently used techniques such as n-gram, support vector machine, and conditional random field approaches to identify language in code-mixed data of social network. Digging deeper the language identification problem as well content of social network we found that irrespective of language, users are more convenient inphoneme/phone-based writing system compared to following the actual writing convention. We also found that hardly few messages are written using standard norms. This article discusses a simple phone-based n -gram method to identify language in text of social network. Obtained result is encouraging and comparable to state-of-the-art of the code-mixed data.