Development of the information system for the Kazakh language preprocessing. Issue 1 (1st January 2021)

• Record Type: Journal Article
• Authors: Akhmed-Zaki, Darkhan; Mansurova, Madina; Madiyeva, Gulmira; Kadyrbek, Nurgali; Kyrgyzbayeva, Marzhan
• Editors: Pham, Duc
• Is Part Of: Cogent engineering. Volume 8:Issue 1(2021)
• Publication Date: 2021-01-01
• Subjects: Corpus linguistics -- morphological parsing -- Kazakh language -- preprocessing tool -- architecture of information system; Engineering -- Periodicals; Technology -- Periodicals; Engineering; Technology; Periodicals; 620
• DOI: 10.1080/23311916.2021.1896418
• Languages: English
• ISSNs: 2331-1916

Abstract

Abstract: The aim of this work is the design and development of linguistic resources and preprocessing tools for the Kazakh language. The media-corpus of the Kazakh language is presented as a linguistic resource, which is available on Al-Farabi Kazakh National University platform. The media-corpus of the Kazakh language consists of texts of news content and is implemented as an information system. The general architecture of an information system for the automatic and reliable collection, storage and analysis of texts in the Kazakh language is described. Three automatic text preprocessing tools for the Kazakh language – word forms generator, morphological analyzer, and morphological disambiguation tool – are presented in the article. The proposed tools can also be applied in the systems of automatic analysis of texts, in creation of other linguistic resources such as thesauri and ontologies.