Cross Knowledge-based Generative Zero-Shot Learning approach with Taxonomy Regularization. (July 2021)

• Record Type: Journal Article
• Authors: Xie, Cheng; Xiang, Hongxin; Zeng, Ting; Yang, Yun; Yu, Beibei; Liu, Qing
• Is Part Of: Neural networks. Volume 139(2021)
• Page Start: 168
• Page End: 178
• Publication Date: 2021-07
• Subjects: Zero-Shot Learning -- Image recognition -- Generative Adversarial Network -- Knowledge engineering; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.02.009
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Although zero-shot learning (ZSL) has an inferential capability of recognizing new classes that have never been seen before, it always faces two fundamental challenges of the cross modality and cross-domain challenges. In order to alleviate these problems, we develop a generative network-based ZSL approach equipped with the proposed Cross Knowledge Learning (CKL) scheme and Taxonomy Regularization (TR). In our approach, the semantic features are taken as inputs, and the output is the synthesized visual features generated from the corresponding semantic features. CKL enables more relevant semantic features to be trained for semantic-to-visual feature embedding in ZSL, while Taxonomy Regularization (TR) significantly improves the intersections with unseen images with more generalized visual features generated from generative network. Extensive experiments on several benchmark datasets (i.e., AwA1, AwA2, CUB, NAB and aPY) show that our approach is superior to these state-of-the-art methods in terms of ZSL image classification and retrieval. Highlights: Cross Knowledge Learning (CKL) is proposed to train more relevant semantic features. Taxonomy Regularization (TR) is designed to generate more generalized visual features. Extensive experiments show our approach performs well for zero-shot learning task. Graphical abstract: