Weakly Supervised Instance Segmentation via Category-aware Centerness Learning with Localization Supervision. (April 2023)

• Record Type: Journal Article
• Authors: Zhang, Jiabin; Su, Hu; He, Yonghao; Zou, Wei
• Is Part Of: Pattern recognition. Volume 136(2023)
• Publication Date: 2023-04
• Subjects: Weakly supervised learning -- Instance segmentation -- Centerness -- Coarse localization annotation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109165
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel two-branch DCNN is constructed to perform instance segmentation under the localization supervision. The centerness branch is to extract high-level semantic information which can present the spatial distribution of object instances. The segmentation branch accomplishes the task of instance segmentation. The entire DCNN is efficiently trained end-to-end even with imprecise localization supervision. Based on the output of centerNess branch, an adaptive strategy is utilized to the proposal generation method, MCG, to improve the computational efficiency. Meanwhile, a rank-based algorithm is proposed to sort the boundary proposals to substitute the ground truth mask. Experiments demonstrate that, satisfied accuracy are achieved by the proposed approach compared with other SOTA approaches. Especially, the approach could be well trained under the supervision of imprecise bounding box or scribble while previous methods could not. This is helpful to further reduce human workload of image labelling. Abstract: Deep convolutional neural networks (DCNN) trained with pixel-level segmentation masks achieve high performance in the task of instance segmentation. The difficulty of acquiring such annotation limits the application and popularization of the DCNN-based approaches. To address the issue, a weakly supervised approach is proposed in the paper which performs instance segmentation only with the supervision of bounding box or coarse localization annotation. A novel DCNN model is constructed which consists of two branches: the centerness branch and the segmentation branch. The former branch is to learn the semantically spatial importance over the areas of object instances under the localization supervision. Object proposals with exact boundaries are automatically generated and are then ranked under the guidance of the output of the centerness branch. The most matched instance proposal is assigned to each object, which is then used to supervise the segmentation branch. The losses are calculated by both the outputs of the two branches and the entire DCNN model is trained end-to-end. Experiments are extensively conducted to verify the effectiveness. With the supervision of precise bounding box annotation, our approach achieves state-of-the-art (SOTA) accuracy in the comparison with recent related works. And in the case of coarse localization annotation, our approach only deduces a slight reduction in accuracy, which significantly outperforms other approaches. The excellent performance demonstrates that our approach would be helpful to further alleviate the workload of image annotation while maintaining competitive accuracy.