A hierarchical sampling based triplet network for fine-grained image classification. (July 2021)

• Record Type: Journal Article
• Authors: He, Guiqing; Li, Feng; Wang, Qiyao; Bai, Zongwen; Xu, Yuelei
• Is Part Of: Pattern recognition. Volume 115(2021)
• Publication Date: 2021-07
• Subjects: Metric learning -- Triplet network -- Layered ontology -- Layered triplet loss -- Multi-task learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107889
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Building layered ontology for 3 different databases to guide the layered knowledge of the network. Hierarchical sampling method for mining more effective hard triplets to get better performance. Acquisition of more discriminative deep feature by building hierarchical structure with semantic knowledge. Combination of hierarchical structure and triplet loss to construct a layered Triplet loss function. Abstract: Deep metric learning leverages well-designed distance measurement and a sample selection strategy to learn a discriminative feature space. Among the various deep metric learning formulations, triplet loss is built based on a 3-tuple that can simultaneously minimise the distance between the items in the positive pair and maximise the distance between those in the negative pair. However, this endeavour requires a critical selection of triplet samples to guide the training process. In this paper, we propose a layered Triplet loss to solve the fine-grained image classification problem. Unlike the existing triplet loss, which selects samples from only a single criterion, we construct the loss function with the 'coarse to fine' scheme. This scheme can separate the coarse-level classes while clustering the fine-level samples within a certain margin. An ontology-based sampling method is proposed to enable the network to mine more reasonable hard triplets. Semantic knowledge is employed to assign the visually similar classes to the same learning task, from which hard triplets can be generated. Finally, the softmax tree classifier is used to classify the hierarchical features. The experimental results on multiple datasets demonstrate the effectiveness of the proposed method.