Simultaneous feature selection and discretization based on mutual information. (July 2019)

• Record Type: Journal Article
• Authors: Sharmin, Sadia; Shoyaib, Mohammad; Ali, Amin Ahsan; Khan, Muhammad Asif Hossain; Chae, Oksam
• Is Part Of: Pattern recognition. Volume 91(2019:Jul.)
• Page Start: 162
• Page End: 174
• Publication Date: 2019-07
• Subjects: Feature selection -- Mutual information -- Bias -- Dynamic discretization; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.02.016
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We address discretization and feature selection jointly with a single criteria. The proposed discretization method is dynamic and independent of classification algorithms. The amount of errors introduced for Relevancy, Redundancy and Complementary Information are derived analytically. It is also analytically shown that Relevancy, Redundancy and Complementary follows χ 2 -distribution. A χ 2 -based search is introduced to select a small set of features and to discretize them with small number of intervals. Abstract: Recently mutual information based feature selection criteria have gained popularity for their superior performances in different applications of pattern recognition and machine learning areas. However, these methods do not consider the correction while computing mutual information for finite samples. Again, finding appropriate discretization of features is often a necessary step prior to feature selection. However, existing researches rarely discuss both discretization and feature selection simultaneously. To solve these issues, Joint Bias corrected Mutual Information (JBMI) is firstly proposed in this paper for feature selection. Secondly, a framework namely modified discretization and feature selection based on mutual information is proposed that incorporates JBMI based feature selection and dynamic discretization, both of which use a χ 2 based searching method. Experimental results on thirty benchmark datasets show that in most of the cases, the proposed methods outperform the state-of-the-art methods.