A zero-shot learning framework via cluster-prototype matching. (April 2022)

• Record Type: Journal Article
• Authors: Zhang, Jing; Li, Qingyong; Geng, YangLi-ao; Wang, Wen; Sun, Wenju; Shi, Chuan; Ding, Zhengming
• Is Part Of: Pattern recognition. Volume 124(2022)
• Publication Date: 2022-04
• Subjects: Zero-shot learning -- Image classification -- Cluster-prototype matching -- Domain shift; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108469
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel Cluster-Prototype Matching (CPM) strategy is proposed to solve the domain shift problem of zero-shot learning. The distribution information of samples in the embedding space, i.e., cluster structure, is utilized to assist in zero-shot learning classification. The CPM strategy can be applied to most existing ZSL models in a plug-and-play style. Extensive experiments show the effectiveness and robustness of the proposed method. Abstract: Given the descriptions of classes, Zero-Shot Learning (ZSL) aims to recognize unseen samples by learning a projection between the visual features of samples and the semantic descriptions (prototypes) of classes from seen data. However, due to the inherent distribution gap between seen and unseen domains, the learned projection is generally biased to seen classes and may produce misleading relationships between unseen samples and prototypes (sample-prototype relationship). To tackle this problem, we propose a Cluster-Prototype Matching (CPM) framework which exploits the distribution information of samples to explore the cluster structure of samples and then use the robust cluster-prototype relationship to correct the biased sample-prototype relationship. Specifically, we first use an iterative cluster generation module to identify the underlying cluster structure of samples based on their embedding features, which are acquired via a basic ZSL model. Then each identified cluster will be matched with a specific class prototype through the Kuhn-Munkres algorithm, based on which we can export a sharp cluster-prototype similarity. Finally, the cluster-prototype similarity is combined with the sample-prototype similarity to determine the class labels of test samples. We apply CPM to five well-established ZSL methods and the experimental results show that CPM can significantly improve the performance of basic models and enable them achieve or beyond the state-of-the-art.