Semi-supervised vector-valued learning: Improved bounds and algorithms. (June 2023)

• Record Type: Journal Article
• Authors: Li, Jian; Liu, Yong; Wang, Weiping
• Is Part Of: Pattern recognition. Volume 138(2023)
• Publication Date: 2023-06
• Subjects: Vector-valued learning -- Semi-supervised learning -- Excess risk bound -- Local rademacher complexity; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109356
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Unified error bounds for vector-valued learning in kernel and linear spaces. Local Rademacher complexity improves the converge rate from O ( 1 / n ) to O ( 1 / n ) . Unlabeled examples improve the converge rate from O ( 1 / n ) to O ( 1 / n + u + 1 / n ) . Efficient semi-supervised vector-valued learning algorithm with random features. Abstract: Vector-valued learning, where the output space admits a vector-valued structure, is an important problem that covers a broad family of important domains, e.g. multi-task learning and transfer learning. Using local Rademacher complexity and unlabeled data, we derive novel semi-supervised excess risk bounds for general vector-valued learning from both kernel perspective and linear perspective. The derived bounds are much sharper than existing ones and the convergence rates are improved from the square root of labeled sample size to the square root of total sample size or directly dependent on labeled sample size. Motivated by our theoretical analysis, we propose a general semi-supervised algorithm for efficiently learning vector-valued functions, incorporating both local Rademacher complexity and Laplacian regularization. Extensive experimental results illustrate the proposed algorithm significantly outperforms the compared methods, which coincides with our theoretical findings.