Kernelized Supervised Laplacian Eigenmap for Visualization and Classification of Multi-Label Data. (March 2022)

• Record Type: Journal Article
• Authors: Tai, Mariko; Kudo, Mineichi; Tanaka, Akira; Imai, Hideyuki; Kimura, Keigo
• Is Part Of: Pattern recognition. Volume 123(2022)
• Publication Date: 2022-03
• Subjects: Supervised Laplacian eigenmaps -- Out-of-sample problem -- Multi-label problems -- Kernel trick -- Separability-guided feature extraction; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108399
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We solves the "out-of-sample problem", inability of handling new data, that previous (supervised) Laplacian eigenmaps have, by a set of linear sums of kernels, and we propose a Kernelized Supervised Laplacian Eigenmap for Multi-Label (KSLE-ML). We show that nonsingularity of Gram matrix is a sufficient condition for this simulation to be exactly the same as Laplacian eigenmaps. We reveal that parameter selection is more important than kernel selection in KSLE-ML experimentally, so that RBF kernel solely can be used for the general purpose. We show a method of separability-guided feature extraction that is based on a high separability of classes in 2D visualization. We confirm empirically the effectiveness of separability-guided feature extraction by showing that the separability is kept well even for mapping of newly arrived samples without class labels in KSLE-ML. We also demonstrate that the effectiveness increases as the number of samples and the mapping dimension increases. Abstract: We had previously proposed a supervised Laplacian eigenmap for visualization (SLE-ML) that can handle multi-label data. In addition, SLE-ML can control the trade-off between the class separability and local structure by a single trade-off parameter. However, SLE-ML cannot transform new data, that is, it has the "out-of-sample" problem. In this paper, we show that this problem is solvable, that is, it is possible to simulate the same transformation perfectly using a set of linear sums of reproducing kernels (KSLE-ML) with a nonsingular Gram matrix. We experimentally showed that the difference between training and testing is not large; thus, a high separability of classes in a low-dimensional space is realizable with KSLE-ML by assigning an appropriate value to the trade-off parameter. This offers the possibility of separability-guided feature extraction for classification. In addition, to optimize the performance of KSLE-ML, we conducted both kernel selection and parameter selection. As a result, it is shown that parameter selection is more important than kernel selection. We experimentally demonstrated the advantage of using KSLE-ML for visualization and for feature extraction compared with a few typical algorithms.