Uncertainty-aware semi-supervised few shot segmentation. (May 2023)

• Record Type: Journal Article
• Authors: Kim, Soopil; Chikontwe, Philip; An, Sion; Park, Sang Hyun
• Is Part Of: Pattern recognition. Volume 137(2023)
• Publication Date: 2023-05
• Subjects: Few shot segmentation -- Meta learning -- Uncertainty estimation -- Semi-supervised learning -- Prototype; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109292
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a new semi-supervised few-shot segmentation (FSS) method that employs additional prototypes from unlabeled images. Our approach can be trained without an additional learning process for unlabeled samples. We propose a novel uncertainty estimation method for prototype-based FSS. Our method can reliably quantify uncertainty without degrading the baseline performance of existing FSS models. Our method shows significant improvement over two baseline methods on two FSS benchmarks, i.e., PASCAL-5i and COCO-20i. Graphical abstract: The inference step of the proposed uncertainty-aware semi-supervised FSS model. Abstract: Few shot segmentation (FSS) aims to learn pixel-level classification of a target object in a query image using only a few annotated support samples. This is challenging as it requires modeling appearance variations of target objects and the diverse visual cues between query and support images with limited information. To address this problem, we propose a semi-supervised FSS strategy that leverages additional prototypes from unlabeled images with uncertainty guided pseudo label refinement. To obtain reliable prototypes from unlabeled images, we meta-train a neural network to jointly predict segmentation and estimate the uncertainty of predictions. We employ the uncertainty estimates to exclude predictions with high degrees of uncertainty for pseudo label construction to obtain additional prototypes from the refined pseudo labels. During inference, query segmentation is predicted using prototypes from both support and unlabeled images including low-level features of the query images. Our approach can easily supplement existing approaches without the requirement of additional training when employing unlabeled samples. Extensive experiments on PASCAL- 5 i and COCO- 20 i demonstrate that our model can effectively remove unreliable predictions to refine pseudo labels and significantly improve upon baseline performance.