Unsupervised feature selection by non-convex regularized self-representation. (1st July 2021)

• Record Type: Journal Article
• Authors: Miao, Jianyu; Ping, Yuan; Chen, Zhensong; Jin, Xiao-Bo; Li, Peijia; Niu, Lingfeng
• Is Part Of: Expert systems with applications. Volume 173(2021)
• Publication Date: 2021-07-01
• Subjects: Unsupervised feature selection -- Self-representation -- Non-convex regularization -- CCCP -- ADMM; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.114643
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: An ℓ 2, 1 - 2 self-representation unsupervised feature selection is proposed. ℓ 2, 1 - 2 is proved to guarantee the sparsity of selection matrix in theory. An iterative CCCP algorithm is designed to tackle the nonconvexity of ℓ 2, 1 - 2 . The global convergence of our CCCP is theoretically analyzed. Extensive experimental results verify the effectiveness of the proposed method. Abstract: Feature selection, as a crucial pre-processing stage in expert and intelligent systems, aims at reducing the dimensionality of the high-dimensional data by selecting the optimal subset from original features set. It can enhance the interpretability, improve learning performance, and increase computational efficiency. In real-world applications, obtaining class labels of data is time consuming and labor intensive, thus unsupervised feature selection is more practically important but correspondingly more challenging. Self-representation learning provides some insights on unsupervised feature selection, whose goal is to identify a representative feature subset so that all the features can be well reconstructed by them. In this paper, we propose a new unsupervised feature selection method by using NOn-conVex Regularized Self-Representation (NOVRSR). Different from most prior researches resorting to pseudo labels of data, NOVRSR exploits importance and relevance of features by self-representation. Moreover, the ℓ 2, 1 - 2 sparse regularization, which is non-convex yet Lipschitz continuous, is enforced on the representation coefficient matrix to perform feature selection. We show in theory that the utilization of ℓ 2, 1 - 2 can guarantee the sparsity of the representation coefficient matrix. In addition, to find the solution of the resulting non-convex formula, we design an iterative algorithm in the framework of ConCave-Convex Procedure (CCCP) and prove that the iterative sequence converges to the stationary point satisfying the first-order optimality condition. An adopted Alternating Direction Method of Multipliers (ADMM) is embedded to solve the sequence of convex subproblems of CCCP efficiently. Extensive experimental studies on real-world datasets demonstrate that the effectiveness of the proposed method.