Multi-attention mutual information distributed framework for few-shot learning. (15th September 2022)

• Record Type: Journal Article
• Authors: Wang, Zhe; Ma, Pingchuan; Chi, Ziqiu; Li, Dongdong; Yang, Hai; Du, Wenli
• Is Part Of: Expert systems with applications. Volume 202(2022)
• Publication Date: 2022-09-15
• Subjects: Few-shot learning -- Attention mechanism -- Mutual learning -- Distributed learning; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.117062
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: The purpose of few-shot learning is to learn a classifier, even if only a limited number of samples are used, a good generalization effect can be achieved. Recently, many methods based on meta-learning learn a large number of multi-classification tasks to train a general classifier to solve this problem. Methods based on metric learning use the distance relationship between labeled samples and unlabeled samples for classification. These methods all have good performance. However, these methods rarely pay attention to the problems of insufficient feature extraction and low training efficiency. To this end, we propose multi-attention mutual information distributed framework for few-shot learning (MAMD). Specifically, we use the attention mechanism to help the feature embedding module extract more representative features. We use multiple attention mechanisms because different attention mechanisms can focus on different features. After multiple attention modules extract features, we use the mutual learning to aggregate the extracted features. The mutual learning method is similar to knowledge distillation. The difference between knowledge distillation and mutual learning is that mutual learning does not require a large network to guide a small network, but two networks learn from each other and progress together. In addition, we use distributed learning to improve the training speed and shorten the consumption of time. We combine distributed learning with few-shot learning for the first time and propose the concept of distributed few-shot learning. It provides a new direction for few-shot learning. We evaluate MAMD on two few-shot learning benchmark datasets and achieve the expected results. Highlights: Mutual learning module based on the multi-attention mechanism. Combine few-shot learning and distributed learning. Propose the concept of distributed few-shot learning for the first time. The method achieves the expected performance on two datasets.