A unified model for the sparse optimal scoring problem. (January 2023)

• Record Type: Journal Article
• Authors: Li, Guoquan; Yang, Linxi; Zhao, Kequan
• Is Part Of: Pattern recognition. Volume 133(2023)
• Publication Date: 2023-01
• Subjects: Optimal scoring -- Linear discriminant analysis -- Feature selection -- ℓq−norm -- Sparseness; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108976
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A unified model for sparse optimal scoring is proposed by employing L u -norm ( 0 ≤ q ≤ 1 ) regular term where L o − norm and( L u ) -norm ( 0 ≤ q ≤ 1 ) will be selected adaptively to find more sparser solutions. We derive an efficient iterative algorithm based on alternating direction method of multipliers (ADMM) for the new formulation. The new proposed method can solve ( L o − ) norm regularized problem directly rather than using convex or nonconvex approximations of ( L o − ) norm. The convergence of the algorithm is analyzed theoretically. Extensive numerical experiments show that our algorithm is efficient not only in classification accuracy but also in sparsity. Abstract: Optimal scoring (OS), an equivalent form of linear discriminant analysis (LDA), is an important supervised learning method and dimensionality reduction tool. However, it is still a challenge for the classical OS on small sample size (SSS) datasets. In this paper, to find sparse discriminant vectors, we propose a unified model for sparse optimal scoring (SOS) by virtue of the generalized ℓ q -norm ( 0 ≤ q ≤ 1 ). To overcome the difficulty in treating the generalized ℓ q -norm, we propose an efficient alternative direction method of multipliers (ADMM), where proximity operator of ℓ q -norm is employed for different q values. Meanwhile, the convergence results of our method are also established. Numerical experiments on artificial and benchmark datasets demonstrate the effectiveness and feasibility of our proposed method.