A Contrastive learning-based Task Adaptation model for few-shot intent recognition. Issue 3 (May 2022)

• Record Type: Journal Article
• Authors: Zhang, Xin; Cai, Fei; Hu, Xuejun; Zheng, Jianming; Chen, Honghui
• Is Part Of: Information processing & management. Volume 59:Issue 3(2022)
• Publication Date: 2022-05
• Subjects: Intent recognition -- Few-shot learning -- Contrastive learning; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2021.102863
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: Few-shot intent recognition aims to identify user's intent from the utterance with limited training data. A considerable number of existing methods mainly rely on the generic knowledge acquired on the base classes to identify the novel classes. Such methods typically ignore the characteristics of each meta task itself, resulting in the inability to make full use of limited given samples when classifying unseen classes. To deal with such issues, we propose a C ontrastive learning-based T ask A daptation model (CTA) for few-shot intent recognition. In detail, we leverage contrastive learning to help achieve task adaptation and make full use of the limited samples of novel classes. First, a self-attention layer is employed in the task adaptation module, which aims to establish interactions between samples of different categories so that new representations are task-specific rather than relying entirely on the base classes. Then, the contrastive-based loss functions and the semantics of the label name are respectively used for reducing the similarity between sample representations in different categories while increasing it in the same categories. Experimental results on a public dataset OOS verify the effectiveness of our proposal by beating the competitive baselines in terms of accuracy. Besides, we conduct the cross-domain experiments on three datasets, i.e., OOS, SNIPS as well as ATIS. We find that CTA gains obvious improvements in terms of accuracy in all cross-domain experiments, indicating that it has a better generalization ability than other competitive baselines in both cross-domain and single-domain settings. Highlights: We employ a CTA model to get the task-unique feature for few-shot intent recognition. We introduce a contrastive-based loss function to well separate different classes. We use the semantics of label name as an anchor feature of each class to fix bias. Our model performs particularly well in multi-domain and cross-domain scenarios.