Large-scale multi-label classification using unknown streaming images. (March 2020)

• Record Type: Journal Article
• Authors: Zhang, Yu; Wang, Yin; Liu, Xu-Ying; Mi, Siya; Zhang, Min-Ling
• Is Part Of: Pattern recognition. Volume 99(2020:Mar.)
• Publication Date: 2020-03
• Subjects: Multi-label image classification -- Recurrent novel-class detector -- Streaming images; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107100
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We proposed to learn the problem of multi-label classification from streaming images with unknown classes in a unified deep learning framework. We proposed to learn a recurrent novel-class detector for novel-class detection, which naturally encodes the relationship in image features and labels. The proposed method is systematically evaluated on large-scale benchmark datasets, which shows its efficacy for practical deployment. Abstract: In this paper, we investigate the large-scale multi-label image classification problem when images with unknown novel classes come in stream during the training stage. It coincides with the practical requirement that usually novel classes are detected and used to update an existing image recognition system. Most existing multi-label image classification methods cannot be directly applied in this scenario, where the training and testing stages must have the same label set. In this paper, we proposed to learn a multi-label classifier and a novel-class detector alternately to solve this problem. The multi-label classifier is learned using a convolutional neural network (CNN) from the images in the known classes. We proposed a recurrent novel-class detector which is learned in the supervised manner to detect the novel class by encoding image features with the multi-label information. In the experiment, our method is evaluated on several large-scale multi-label benchmarks including MS COCO. The results show the proposed method is comparable to most existing multi-label image classification methods, which validate its efficacy when encountering streaming images with unknown classes.