SAPENet: Self-Attention based Prototype Enhancement Network for Few-shot Learning. (March 2023)

• Record Type: Journal Article
• Authors: Huang, Xilang; Choi, Seon Han
• Is Part Of: Pattern recognition. Volume 135(2023)
• Publication Date: 2023-03
• Subjects: Few-shot learning -- Multi-head self-attention mechanism -- Image classification -- k-Nearest neighbor; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109170
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We present a self-attention based few-shot learning framework to address the issue of mean prototypes learning redundant information. An intra-class attention block is developed to efficiently capture the informative channel features among intra-class sample features by exploring the relative similarities between them. We conduct numerous experiments on three few-shot learning benchmark datasets. The experimental results show that our method largely improves the Prototypical network and outperforms the related state-of-the-art methods that concentrate on metric-based learning and attention mechanism. Abstract: Few-shot learning considers the problem of learning unseen categories given only a few labeled samples. As one of the most popular few-shot learning approaches, Prototypical Networks have received considerable attention owing to their simplicity and efficiency. However, a class prototype is typically obtained by averaging a few labeled samples belonging to the same class, which treats the samples as equally important and is thus prone to learning redundant features. Herein, we propose a self-attention based prototype enhancement network (SAPENet) to obtain a more representative prototype for each class. SAPENet utilizes multi-head self-attention mechanisms to selectively augment discriminative features in each sample feature map, and generates channel attention maps between intra-class sample features to attentively retain informative channel features for that class. The augmented feature maps and attention maps are finally fused to obtain representative class prototypes. Thereafter, a local descriptor-based metric module is employed to fully exploit the channel information of the prototypes by searching k similar local descriptors of the prototype for each local descriptor in the unlabeled samples for classification. We performed experiments on multiple benchmark datasets: miniImageNet, tieredImageNet, and CUB-200-2011. The experimental results on these datasets show that SAPENet achieves a considerable improvement compared to Prototypical Networks and also outperforms related state-of-the-art methods.