Weakly supervised image classification and pointwise localization with graph convolutional networks. (January 2021)

• Record Type: Journal Article
• Authors: Liu, Yongsheng; Chen, Wenyu; Qu, Hong; Mahmud, S.M. Hasan; Miao, Kebin
• Is Part Of: Pattern recognition. Volume 109(2021)
• Publication Date: 2021-01
• Subjects: Deep learning -- Learning systems -- Convolutional neural networks -- Predictive models -- Image classification -- Graph theory; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107596
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A new deep learning framework is proposed in this paper, which can leverage the object label inter-dependent for weakly supervised learning. We creatively introduce a novel initialization method of label embeddings for inter-relationships learning. Ablation studies of our model are provided in detail. Our models can be applied to object classification and weakly supervised pointwise object localization. Abstract: In computer vision, the research community has been looking to how to benefit from weakly supervised learning that utilizes easily obtained image-level labels to train neural network models. The existing deep convolutional neural networks for weakly supervised learning, however, generally do not fully exploit the label dependencies in an image. To make full use of this information, in this paper, we propose a new framework for weakly supervised learning of deep convolutional neural networks, introducing graph convolutional networks to capture the semantic label co-occurrence in an image. Moreover, we propose a novel initialization method for label embedding in graph convolutional networks, which enables a smoother optimization for interrelationships learning. Extensive experiments and comparisons on four public benchmark datasets (PASCAL VOC 2007, PASCAL VOC 2012, Microsoft COCO, and NUS-WIDE) show the superior performance of our approach in both image classification and weakly supervised pointwise object localization. These results lead us to conclude that the label dependencies in the input image can provide valuable evidence for learning strongly localized features.