Learning multi-layer coarse-to-fine representations for large-scale image classification. (July 2019) - Zhang, Ji; Mei, Kuizhi; Zheng, Yu; Fan, Jianping
Highlights: Inter-category visual and semantic correlations are exploited. Large numbers of structural image classes are organized hierarchically. Hierarchical multi-task SVMs are trained over the visual-semantic tree. The visual-semantic tree and CNNs are integrated as another framework. We perform our experiments on 10k image categories for algorithm evaluation. Abstract: Recent studies on large-scale image classification mainly focus on categorizing images into 1000 object classes, and all these 1000 object classes are atomic and mutually exclusive in the semantic space. However, for a much larger set of image categories (such as the ImageNet 10k dataset), some of them may come from the high-level (non-leaf) nodes of the concept ontology and could contain some other lower-level categories semantically. The research that classifies images into large numbers of image categories with such inter-category subsumption correlations has received rare attention. In this paper, a Visual-Semantic Tree is learned to organize 10k image categories hierarchically in a coarse-to-fine fashion, where both the inter-category visual similarities and inter-category semantic correlations are seamlessly integrated for tree construction. Additionally, a deep learning method is developed by integrating the Visual-Semantic Tree with deep CNNs to learn more discriminative tree classifiers for large-scale image classification. Our experimental results have demonstrated that the proposed Visual-Semantic Tree can effectively organize large-scale structural image categories and significantly boost the classification accuracy rates for both atomic image categories and high-level image categories.