Feature selection method with joint maximal information entropy between features and class. (May 2018)

• Record Type: Journal Article
• Authors: Zheng, Kangfeng; Wang, Xiujuan
• Is Part Of: Pattern recognition. Volume 77(2018:May)
• Page Start: 20
• Page End: 29
• Publication Date: 2018-05
• Subjects: BPSO -- Entropy -- Feature selection -- Maximal information coefficient; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.12.008
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A new metric (joint maximal information entropy (JMIE)) is defined to measure a feature subset. A new feature selection method combining the joint maximal information entropy among features (FS-JMIE) and binary particle swarm optimization (BPSO) algorithm is proposed in this paper. Experimental results on 5 UCI datasets show the efficiency of the proposed feature selection method. The proposed method manifests advantage in feature selection with multiple classes. FS-JMIE shows higher consistency and better time-efficiency than BPSO-SVM algorithm. Abstract: Feature selection remains a popular method for quantity reduction of attributes of high-dimensional data, to reduce computational costs in classifications. A new feature selection method based on the joint maximal information entropy between features and class (FS-JMIE) is proposed in this paper. Firstly, the joint maximal information entropy (JMIE) is defined to measure a feature subset. Next, a binary particle swarm optimization (BPSO) algorithm is introduced to search the optimal feature subset. Finally, classification is performed on UCI corpora to verify the performance of our proposed method compared to the traditional mutual information (MI) method, CHI method, as well as a binary version of particle swarm optimization-support vector machines (BPSO-SVMs) feature selection. Experiments show that FS-JMIE achieves an equal or better performance than MI, CHI, and BPSO-SVM. Further, FS-JMIE manifests relatively better robustness to the number of classes. Moreover, the method shows higher consistency and better time-efficiency than BPSO-SVM.