Jointly sparse least square support vector machine. (July 2022) - Chen, Xi; Lai, Zhihui
Abstract: Least square support vector machine (LS-SVM) is extended from support vector machine (SVM) for binary classification problems. However, it may suffer from the small sample size (SSS) problem when the sample size is much smaller than the number of features. Motivated by the dimensionality reduction and feature selection methods, we introduce L 2, 1 -norm into LS-SVM to design a novel classification algorithm called jointly sparse LS-SVM (JS-LSSVM). JS-LSSVM minimizes the L 2, 1 -norm regularization on the projection matrix with orthogonal constraint, which is used to project the samples into an optimal low-dimensional subspace, where the derived LS-SVM can obtain the best performance. This projection matrix releases the least square problem in primal space and allows us to select features with joint sparsity. Besides, we propose an iterative algorithm to solve the optimization problem, which guarantees the convergence of JS-LSSVM. The experiments also show the superior performance of JS-LSSVM on many datasets. The proposed method has at least 1% improvement to the conventional methods. Graphical abstract: Highlights: A seamless framework combining sparsity learning, feature extraction, and classification. A novel iterative algorithm is designed to solve the optimization problem. The small sample size problem is well addressed.