A combination of fuzzy similarity measures and fuzzy entropy measures for supervised feature selection. (15th November 2018)

• Record Type: Journal Article
• Authors: Lohrmann, Christoph; Luukka, Pasi; Jablonska-Sabuka, Matylda; Kauranne, Tuomo
• Is Part Of: Expert systems with applications. Volume 110(2018)
• Page Start: 216
• Page End: 236
• Publication Date: 2018-11-15
• Subjects: Feature ranking -- Filter method -- Wrapper method -- Machine learning -- ReliefF; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2018.06.002
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Combination of similarity and entropy measures (FSAE) for feature selection. Includes scale-factor for inter-class distances to scale entropy values. The model is tested on five medical data sets from UCI Machine Learning Depository. Feature selection algorithm results in high classification accuracy on given data sets. Abstract: Large amounts of information and various features are in many machine learning applications available, or easily obtainable. However, their quality is potentially low and greater volumes of information are not always beneficial for machine learning, for instance, when not all available features in a data set are relevant for the classification task and for understanding the studied phenomenon. Feature selection aims at determining a subset of features that represents the data well, gives accurate classification results and reduces the impact of noise on the classification performance. In this paper, we propose a filter feature ranking method for feature selection based on fuzzy similarity and entropy measures (FSAE), which is an adaptation of the idea used for the wrapper function by Luukka (2011) and has an additional scaling factor. The scaling factor to the feature and class-specific entropy values that is implemented, accounts for the distance between the ideal vectors for each class. Moreover, a wrapper version of the FSAE with a similarity classifier is presented as well. The feature selection method is tested on five medical data sets: dermatology, chronic kidney disease, breast cancer, diabetic retinopathy and horse colic. The wrapper version of FSAE is compared to the wrapper introduced by Luukka (2011) and shows at least as accurate results with often considerably fewer features. In the comparison with ReliefF, Laplacian score, Fisher score and the filter version of Luukka (2011), the FSAE filter in general achieves competitive mean accuracies and results for one medical data set, the breast cancer Wisconsin data set, together with the Laplacian score in the best results over all possible feature removals.