Adaptive open domain recognition by coarse-to-fine prototype-based network. (August 2022)

• Record Type: Journal Article
• Authors: Yuan, Yuan; He, Xinxing; Jiang, Zhiyu
• Is Part Of: Pattern recognition. Volume 128(2022)
• Publication Date: 2022-08
• Subjects: Open domain recognition -- Image classification -- Adaptive openness -- Prototype learning -- Unknown class recognition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108657
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: For open domain recognition task, the overlap between known and unknown categorys, which is defined as openness, can affect the recognition model a lot. In this work, a realistic setting termed named as Adaptive Open Domain Recognition is firstly introduced to consider this issue. Fusion Information Guided Feature Prototype Generation is proposed to obtain more abundant and accurate descriptive information. FGPG is more robust to various openness since it directly constructs the interaction between visual features and fused category semantics through generative adversarial networks. Class-Aware Feature Prototype Alignment is proposed to align the global feature prototype between two domains. The global feature prototype is adaptively updated in each episode to suppress the negative effects of false pseudo-label and misaligned categories, and further adapt to various openness. Abstract: Open domain recognition has attracted great attention in recent two years, which aims to assign a specific identification for each target sample in the presence of large domain discrepancy both in label space and data distributions. Most existing approaches rely on abundant prior information about the relationship of the label sets between the source and the target domain, which is a great limitation for their applications in practical wild. In this paper, a new Adaptive Open Domain Recognition (AODR) task is introduced, which can generalize to various openness and requires no prior information on the label set. To achieve this adaptive transfer task, a two-stage Progressive Adaptation Network is designed, whose learning process consists of multiple episodes. Each episode is performed to simulate an AODR task. Through training and refining multiple episodes, the basic model has progressively accumulated wealthy experience on predicting unseen categories in the presence of large domain discrepancy, which will well generalize to various openness. More specifically, Fusion Information Guided Feature Prototype Generation module is proposed to synthesize visual feature prototype conditioned on category semantic prototype in training stage. Further, Class-Aware Feature Prototype Alignment module is designed in refining stage to align the global feature prototype for each class between two domains. Experimental results verify that the proposed model not only has superiority on classifying the image instances of known and unknown classes, but also well adapts to various openness.