Sparse Support Matrix Machine. (April 2018)

• Record Type: Journal Article
• Authors: Zheng, Qingqing; Zhu, Fengyuan; Qin, Jing; Chen, Badong; Heng, Pheng-Ann
• Is Part Of: Pattern recognition. Volume 76(2018:Apr.)
• Page Start: 715
• Page End: 726
• Publication Date: 2018-04
• Subjects: Classification -- Support vector machine -- Matrix analysis -- Sparse -- Low rank; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.10.003
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel matrix classifier to simultaneously leverage the structural information within matrices and select useful features. We regularize the combination of nuclear norm and l1 norm of the regression matrix and develop an efficient solver based on GFB splitting framework. We also provide a theoretical guarantee for the global convergence and analyze the excess risk statistically. We extensively evaluate the proposed SSMM on four real datasets. The results show that SSMM achieves competitive performance. Abstract: Modern technologies have been producing data with complex intrinsic structures, which can be naturally represented as two-dimensional matrices, such as gray digital images, and electroencephalography (EEG) signals. When processing these data for classification, traditional classifiers, such as support vector machine (SVM) and logistic regression, have to reshape each input matrix into a feature vector, resulting in the loss of structural information. In contrast, modern classification methods such as support matrix machine capture these structures by regularizing the regression matrix to be low-rank. These methods assume that all entities within each input matrix can serve as the explanatory features for its label. However, in real-world applications, many features are redundant and useless for certain classification tasks, thus it is important to perform feature selection to filter out redundant features for more interpretable modeling. In this paper, we tackle this issue, and propose a novel classification technique called Sparse Support Matrix Machine (SSMM), which is favored for taking both the intrinsic structure of each input matrix and feature selection into consideration simultaneously. The proposed SSMM is defined as a hinge loss for model fitting, with a new regularization on the regression matrix. Specifically, the new regularization term is a linear combination of nuclear norm and ℓ1 norm, to consider the low-rank property and sparse property respectively. The resulting optimization problem is convex, and motivates us to propose a novel and efficient generalized forward-backward algorithm for solving it. To evaluate the effectiveness of our method, we conduct comparative studies on the applications of both image and EEG data classification problems. Our approach achieves state-of-the-art performance consistently. It shows the promise of our SSMM method on real-world applications.