Data generalisation with k-means for scalable data mining. Issue 2 (1st January 2012)

• Record Type: Journal Article
• Authors: Siddiky, F.A.; Kabir, Faisal; Rahman, S.M. Monzurur
• Is Part Of: International journal of knowledge engineering and data mining. Volume 2:Issue 2/3(2012)
• Page Start: 215
• Page End: 235
• Publication Date: 2012-01-01
• Subjects: data mining -- scalability -- CRISP-DM -- SEMMA classification -- decision tree -- K-means -- Bayesian; Knowledge representation (Information theory) -- Periodicals; Data mining -- Periodicals; 006.305
• Languages: English
• ISSNs: 1755-2087

Abstract

A classification paradigm is a data mining framework containing all the concepts extracted from the training dataset to discriminate one class from other classes existing in data. Most of classification frameworks aim to provide a solution where either entire dataset is considered or a fractional dataset is considered. When a classification framework considers whole dataset then the algorithm may become unusable because of the inherent un-scalable nature of the algorithm itself as well as the rapid growing nature of training dataset. The alternative way of making classification usable is to select a small portion of the training dataset. This limits the entire representation of training dataset in algorithm which results to poor performance of classification accuracy when unseen data is presented to the classification framework. Our paper first addresses these problems inherent in classifiers and then proposes a framework using k-means and C4.5 classification algorithm as the solution. The framework is scalable as it can classify any dataset irrespective of the size with significant accuracy rate.