High-order conditional mutual information maximization for dealing with high-order dependencies in feature selection. (November 2022)

• Record Type: Journal Article
• Authors: Souza, Francisco; Premebida, Cristiano; Araújo, Rui
• Is Part Of: Pattern recognition. Volume 131(2022)
• Publication Date: 2022-11
• Subjects: Feature selection -- Mutual information -- Information theory -- Pattern recognition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108895
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Mutual information feature selection method based on conditional mutual information. Novel mutual information feature selection algorithm, exploring high order dependencies. The proposed method allows seeing the mutual information feature selection with highorder dependencies with clear interpretation. Compared with different methods over 20 benchmark datasets, the proposed method reached the best results. The proposed method is faster and accurate than other feature selection methods that explore high order dependencies. Abstract: This paper presents a novel feature selection method based on the conditional mutual information (CMI). The proposed High Order Conditional Mutual Information Maximization (HOCMIM) method incorporates high order dependencies into the feature selection procedure and has a straightforward interpretation due to its bottom-up derivation. The HOCMIM is derived from the CMI's chain expansion and expressed as a maximization optimization problem. The maximization problem is solved using a greedy search procedure, which speeds up the entire feature selection process. The experiments are run on a set of benchmark datasets (20 in total). The HOCMIM is compared with eighteen state-of-the-art feature selection algorithms, from the results of two supervised learning classifiers (Support Vector Machine and K-Nearest Neighbor). The HOCMIM achieves the best results in terms of accuracy and shows to be faster than high order feature selection counterparts.