Learning multi-layer coarse-to-fine representations for large-scale image classification. (July 2019)

• Record Type: Journal Article
• Authors: Zhang, Ji; Mei, Kuizhi; Zheng, Yu; Fan, Jianping
• Is Part Of: Pattern recognition. Volume 91(2019:Jul.)
• Page Start: 175
• Page End: 189
• Publication Date: 2019-07
• Subjects: Visual-semantic tree -- Inter-category correlation -- Multi-task learning -- Deep convolutional neural network -- Large-scale image classification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.02.024
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Inter-category visual and semantic correlations are exploited. Large numbers of structural image classes are organized hierarchically. Hierarchical multi-task SVMs are trained over the visual-semantic tree. The visual-semantic tree and CNNs are integrated as another framework. We perform our experiments on 10k image categories for algorithm evaluation. Abstract: Recent studies on large-scale image classification mainly focus on categorizing images into 1000 object classes, and all these 1000 object classes are atomic and mutually exclusive in the semantic space. However, for a much larger set of image categories (such as the ImageNet 10k dataset), some of them may come from the high-level (non-leaf) nodes of the concept ontology and could contain some other lower-level categories semantically. The research that classifies images into large numbers of image categories with such inter-category subsumption correlations has received rare attention. In this paper, a Visual-Semantic Tree is learned to organize 10k image categories hierarchically in a coarse-to-fine fashion, where both the inter-category visual similarities and inter-category semantic correlations are seamlessly integrated for tree construction. Additionally, a deep learning method is developed by integrating the Visual-Semantic Tree with deep CNNs to learn more discriminative tree classifiers for large-scale image classification. Our experimental results have demonstrated that the proposed Visual-Semantic Tree can effectively organize large-scale structural image categories and significantly boost the classification accuracy rates for both atomic image categories and high-level image categories.