SaberNet: Self-attention based effective relation network for few-shot learning. (January 2023)

• Record Type: Journal Article
• Authors: Li, Zijun; Hu, Zhengping; Luo, Weiwei; Hu, Xiao
• Is Part Of: Pattern recognition. Volume 133(2023)
• Publication Date: 2023-01
• Subjects: Few-shot learning -- Feature representation -- Task analysis -- Transformers; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109024
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We design a novel Self-attention Based Effective Relation Network for few-shot learning and leverage relations not only from local details in feature extraction, but also from support samples and from prototype-query pair channels. We argue the insufficiency of conventional feature extraction in few-shot learning and demonstrate the effectiveness of self-attention in feature extraction. The proposed SaberNet can infer feature relations and model spatial long-range dependencies across features. Extensive experiments and analyses demonstrate the effectiveness of the proposed framework. And Saber network achieves superior performance over other state-of-the-art methods on three challenging datasets. Moreover, we present a simple and powerful baseline to investigate the effect of the backbone in few-shot learning. Abstract: Few-shot learning is an essential and challenging field in machine learning since the agent needs to learn novel concepts with a few data. Recent methods aim to learn comparison or relation between query and support samples to tackle few-shot tasks but have not exceeded human performance and made full use of relations in few-shot tasks. Humans can recognize multiple variants of objects located anywhere in images and compare the relation among learned instances. Inspired by the human learning mechanism, we explore the definition of relations in relation networks and propose self-attention relation modules for feature and learning ability. First, we introduce vision self-attention to generate and purify features in few-shot learning. The comparison of different patches leads the backbone to infer relations between local features, which enforces feature extraction focus on more details. Second, we propose task-specific feature augmentation modules to infer relations and weight different contributions of components in few-shot tasks. The proposed SaberNet is conceptually simple and empirically powerful. Its performance surpasses the baseline a great margin, including pushing 5-way 1-shot CUB accuracy to 89.75% (12.73% absolute improvement), Cars to 76.71% (12.99% absolute improvement) and Flowers to 84.33% (7.67% absolute improvement).