Feature weight estimation based on dynamic representation and neighbor sparse reconstruction. (September 2018)

• Record Type: Journal Article
• Authors: Huang, Xiaojuan; Zhang, Li; Wang, Bangjun; Zhang, Zhao; Li, Fanzhang
• Is Part Of: Pattern recognition. Volume 81(2018:Sep.)
• Page Start: 388
• Page End: 403
• Publication Date: 2018-09
• Subjects: Feature weighting -- Feature selection -- Relief -- Sparse learning -- Local hyperplane -- l1 regularization -- Classification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2018.03.014
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a new dynamic representation framework for feature weight estimation, which redefines the optimization problem. Using gradient ascent method, we provide an effective method to solve the optimization problem of DRNSR-Relief and can guarantee its convergence. A novel neighbor sparse reconstruction method is proposed for represent neighbors of the given samples. Abstract: Relief-like algorithms have been widely used as feature selection to reduce the dimension of high-dimensional data which involves thousands of irrelevant variables because of their low computational cost and high accuracy. Classical Relief algorithms have not exactly shown the dynamic procedure that updates weight iteratively. This paper proposes an innovative feature weight estimation method, called dynamic representation and neighbor sparse reconstruction-based Relief (DRNSR-Relief). Similar to the classical Relief algorithms, the goal of DRNSR-Relief is to maximize the expected margin in the weighted feature space. A dynamic representation framework is introduced to show the dynamic relationship between the expected margin vector and the weight vector. To achieve better neighbor reconstruction, DRNSR-Relief decomposes a nonlinear problem into a set of locally linear ones through local hyperplane with l 1 regularization and then estimates feature weights in a large margin framework. With the help of gradient ascent method, we can guarantee the convergence of DRNSR-Relief. To demonstrate the validity and the effectiveness of our formulation for feature selection in supervised learning, we perform extensive experiments on synthetic and real-world datasets. Experimental results indicate that DRNSR-Relief is very promising.