A conditional-weight joint relevance metric for feature relevancy term. (November 2021)

• Record Type: Journal Article
• Authors: Zhang, Ping; Gao, Wanfu; Hu, Juncheng; Li, Yonghao
• Is Part Of: Engineering applications of artificial intelligence. Volume 106(2021)
• Publication Date: 2021-11
• Subjects: Machine learning -- Feature selection -- Information theory -- Conditional-Weight Joint Relevance; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2021.104481
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: Feature selection is an important preprocessing operation in the fields of machine learning and data mining. Information theory is widely used in feature selection methods because it can measure linear and nonlinear correlations among variables. Traditional information theory-based feature selection methods intend to maximize feature relevancy while minimizing feature redundancy. However, previous feature selection methods focus on either the effect of candidate features or the effect of already-selected features on the feature relevancy. In fact, both candidate features and already-selected features offer important classification information in the design of feature relevancy term. To avoid this problem, we extract useful classification information from joint mutual information to design a novel feature relevancy term named Conditional-Weight Joint Relevance (CWJR). Based on CWJR, we propose a novel feature selection method named Feature Selection considering Conditional-Weight Joint Relevance (CWJR-FS). Additionally, to distinguish the differences between our method and previous methods, we divide information theory-based feature selection methods into two categories: linear-based feature selection methods and nonlinear-based feature selection methods. Finally, our method is compared to seven linear-based methods and four nonlinear-based methods on 19 benchmark data sets. The experimental results demonstrate that CWJR-FS outperforms the compared methods in terms of the average classification accuracy, AUC and F1 score. Highlights: We design a new feature relevancy term. The new feature relevance term includes comprehensive classification information. A new feature selection method named CWJR-FS is proposed. We divide feature selection methods into two categories. CWJR-FS achieves the best classification performance among 11 methods.