Jointly sparse least square support vector machine. (July 2022)

• Record Type: Journal Article
• Authors: Chen, Xi; Lai, Zhihui
• Is Part Of: Computers & electrical engineering. Volume 101(2022)
• Publication Date: 2022-07
• Subjects: Least square support vector machine -- Feature extraction -- Dimensionality reduction -- Subspace learning -- L2, 1-norm sparsity; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108078
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: Least square support vector machine (LS-SVM) is extended from support vector machine (SVM) for binary classification problems. However, it may suffer from the small sample size (SSS) problem when the sample size is much smaller than the number of features. Motivated by the dimensionality reduction and feature selection methods, we introduce L 2, 1 -norm into LS-SVM to design a novel classification algorithm called jointly sparse LS-SVM (JS-LSSVM). JS-LSSVM minimizes the L 2, 1 -norm regularization on the projection matrix with orthogonal constraint, which is used to project the samples into an optimal low-dimensional subspace, where the derived LS-SVM can obtain the best performance. This projection matrix releases the least square problem in primal space and allows us to select features with joint sparsity. Besides, we propose an iterative algorithm to solve the optimization problem, which guarantees the convergence of JS-LSSVM. The experiments also show the superior performance of JS-LSSVM on many datasets. The proposed method has at least 1% improvement to the conventional methods. Graphical abstract: Highlights: A seamless framework combining sparsity learning, feature extraction, and classification. A novel iterative algorithm is designed to solve the optimization problem. The small sample size problem is well addressed.