Local structured feature learning with dynamic maximum entropy graph. (March 2021)

• Record Type: Journal Article
• Authors: Wang, Zheng; Nie, Feiping; Wang, Rong; Yang, Hui; Li, Xuelong
• Is Part Of: Pattern recognition. Volume 111(2021)
• Publication Date: 2021-03
• Subjects: Supervised dimensionality reduction -- Local structured feature learning -- ℓ0-Norm constraint optimization -- Dynamic maximum entropy graph; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107673
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We derive a more discriminative LDA which generates the whitening transformation, and it can minimize the scatter within same samples while keeping the scatter of total samples unchanged simultaneously. Proposed model adaptively selects k neighbors for each data point by imposing L0-norm constraint on similarity matrix. Thus, the graph constructed in our method is k-connected which is more sensitive to local structure of data than full-connected graph in above methods. Proposed model learns the similarity matrix and the transformation matrix simultaneously such that the kNN graph is dynamically updated. In such way, the neighborships of each sample can be found in the optimal subspace rather than in original space. Additionally, we impose a maximum entropy regularization on similarity matrix so that the trivial solution can be avoided naturally. An efficient iterative optimization algorithm is presented to solve proposed problem with L0-norm constraint, and a strict proof of convergence is provided as well. Experiments conducted on synthetic and several real-world data sets demonstrate the superiority of proposed DMEG compared to related state-of-the-art methods on classification task. Abstract: In recent years, Linear Discriminant Analysis (LDA) has seen huge adoption in data mining applications. Due to its globality, it is incompetent to handle multimodal data. Besides, most of LDA's variants learn the projection matrix based on the pre-defined similarity matrix, which is easily affected by noisy and irrelevant features. To address above two issues, a novel local structured feature learning with Dynamic Maximum Entropy Graph (DMEG) method is developed which firstly develops a more discriminative LDA with whitening constraint that can minimize the within-class scatter while keeping the total samples scatter unchanged simultaneously. Second, for exploring the local structure of data, the ℓ0 -norm constraint is imposed on similarity matrix to ensure the k connectivity on graph. More importantly, proposed model learns the similarity and projection matrix simultaneously to ensure that the neighborships can be found in the optimal subspace where the noise have been removed already. Moreover, a maximum entropy regularization is employed to reinforce the discriminability of graph and avoid the trivial solution. Last but not least, an efficient iterative optimization algorithm is provided to optimize proposed model with a NP-hard constraint. Extensive experiments conducted on synthetic and several real-world data sets demonstrate the efficiency in classification task and robustness to noise of proposed method.