Weakly-supervised semantic segmentation with saliency and incremental supervision updating. (July 2021)

• Record Type: Journal Article
• Authors: Luo, Wenfeng; Yang, Meng; Zheng, Weishi
• Is Part Of: Pattern recognition. Volume 115(2021)
• Publication Date: 2021-07
• Subjects: Weakly-supervised -- Semantic segmentation -- Convolution neural networks; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107858
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A new way of weakly-supervised learning with the joint guidance of saliency prior and classification information to effectively obtain better initial supervision for semantic segmentation. More foreground regions are discovered through an incremental supervision updating procedure, which is performed along with the training of segmentation 80 and guarantees to increase the number of foreground pixels. Extensive experiments on two benchmark segmentation datasets verifies the effectiveness of the proposed method for weakly-supervised semantic segmentation. We achieve mIoU of 62.5% and 62.7% on PASCAL VOC val and test set, respectively. Abstract: Weakly-supervised semantic segmentation aims at tackling the dense labeling task using weak supervision so as to reduce human annotation efforts. For weakly-supervised semantic segmentation using only image-level annotation, we propose a novel model of Learning with Saliency and Incremental Supervision Updating (LSISU), in which both the guidances of saliency prior and class information are jointly used and the segmentation supervision is dynamically updated. In the proposed LSISU, we present an image saliency objective complementary to classification loss, by which the trained weakly-supervised deep network can effectively deal with object co-occurrence problem. Meanwhile, we make full use of the class-wise pooling strategy to generate initial mask estimation of high quality. Given an initial annotation, a segmentation network is learned along with incremental supervision updating, which plays a role of region expansion and corrects the falsely estimated supervision for training images. The incremental supervision updating is performed on the fly and involves repeated usage of a fully connected conditional random field algorithm. LSISU achieves superior segmentation performance in terms of mIoU metric on benchmark datasets, which are 62.5% on the PASCAL VOC 2012 test set and 30.1% on the COCO val set.