Adaptive pairing of classifier and imputation methods based on the characteristics of missing values in data sets. (15th March 2016)

• Record Type: Journal Article
• Authors: Sim, Jaemun; Kwon, Ohbyung; Lee, Kun Chang
• Is Part Of: Expert systems with applications. Volume 46(2016)
• Page Start: 485
• Page End: 493
• Publication Date: 2016-03-15
• Subjects: Classification algorithms -- Imputation methods -- Case-based reasoning -- Experiments; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2015.11.004
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Selection of the optimal combination of imputation method and classifier is very costly. A novel method of automatic, adaptive selection of the optimal combination, AMCI, is proposed. Successfully demonstrate the superiority of the proposed method with multiple data sets. The results also suggest that AMCI is scalable: good for bid data analytics and IoT applications. Abstract: Classifiers and imputation methods have played crucial parts in the field of big data analytics. Especially, when using data sets characterized by horizontal scattering, vertical scattering, level of spread, compound metric, imbalance ratio and missing ratio, how to combine those classifiers and imputation methods will lead to significantly different performance. Therefore, it is essential that the characteristics of data sets must be identified in advance to facilitate selection of the optimal combination of imputation methods and classifiers. However, this is a very costly process. The purpose of this paper is to propose a novel method of automatic, adaptive selection of the optimal combination of classifier and imputation method on the basis of features of a given data set. The proposed method turned out to successfully demonstrate the superiority in performance evaluations with multiple data sets. The decision makers in big data analytics could greatly benefit from the proposed method when it comes to dealing with data set in which the distribution of missing data varies in real time.