Improve feature selection method of web page language identification using fuzzy ARTMAP. (16th November 2010)

• Record Type: Journal Article
• Authors: Ng, Choon-Ching; Selamat, Ali
• Is Part Of: International journal of intelligent information and database systems. Volume 4:Number 6(2010)
• Page Start: 629
• Page End: 642
• Publication Date: 2010-11-16
• Subjects: feature selection -- N-grams frequency -- fuzzy ARTMAP -- web pages -- language identification -- web page languages -- language translation -- web page analysis -- reliability -- neural networks -- adaptive resonance theory; Database management -- Computer programs -- Periodicals; Information retrieval -- Computer programs -- Periodicals; Information storage and retrieval systems -- Computer programs -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Intelligent agents (Computer software) -- Periodicals; 006.33
• Languages: English
• ISSNs: 1751-5858

Abstract

The information available in languages other than English on the World Wide Web and global information systems is increasing significantly. Different languages can be produced by using one particular script such as Arabic, Persian, Urdu and Pashto that use Arabic script letters. The issue is how to produce reliable features of a web page that is to undergo language identification. Incorrectly identifying the language results in garbled translations as well as faulty and incomplete analyses. The aim of this study is to enhance the effectiveness of feature selection method of web page language identification. We have investigated total N-grams, N-grams frequency, N-grams frequency document frequency, and N-grams frequency inverse document frequency of web page language identification. From the experimental results, it is proven that N-grams frequency gives the most promising result compared to other feature selection methods.