Scatter matrix decomposition for jointly sparse learning. (August 2023)

• Record Type: Journal Article
• Authors: Mo, Dongmei; Lai, Zhihui; Zhou, Jie; Qinghua, Hu
• Is Part Of: Pattern recognition. Volume 140(2023)
• Publication Date: 2023-08
• Subjects: Feature extraction -- Pattern recognition -- Classification -- Linear discriminant analysis -- Joint sparsity; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109485
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: This paper solves the problem of orthogonal linear discriminant analysis (OLDA) from the novel viewpoint of scatter matrix orthogonal decomposition. The method can obtain approximately orthogonal sparse discriminative vectors for dimensionality reduction and jointly sparse feature extraction. Theoretical analysis shows that OLDA can be derived by the constrained scatter matrix decomposition. The method outperforms several well-known LDA-based and sparse learning methods on four data sets (i.e., COIL100, USPS, ICADAR2003 and CMU PIE). Abstract: Orthogonal Linear Discriminant Analysis (OLDA) based on generalized Eigen-equation is widely used in the field of computer vision and pattern recognition. However, the performance of OLDA for feature extraction and classification needs to be improved as it lacks sparsity for better interpretation of the features. Moreover, computing the orthogonal sparse projections based on LDA is very difficult and is still unsolved. To solve these problems, in this paper, we propose a method called Jointly Sparse Orthogonal Linear Discriminant Analysis (JSOLDA). Different from the existing OLDA, JSOLDA is proposed from a novel viewpoint of scatter matrix decomposition. Theoretical analysis shows that OLDA can be derived by the constrained scatter matrix decomposition. In addition, by imposing L2, 1 -norm on the penalty term, the proposed JSOLDA can obtain the jointly sparse orthogonal projections to perform feature extraction. We also design an iterative algorithm to obtain the optimal solution. Systematic theoretical analysis between the OLDA and JSOLDA are uncovered. Both of convergence and computational complexity are also discussed. Experimental results on four data sets (i.e., COIL100, USPS, ICADAR2003 and CMU PIE) indicate that JSOLDA outperforms several well-known LDA-based and L2, 1 -norm based methods.