Discriminative information restoration and extraction for weakly supervised low-resolution fine-grained image recognition. (July 2022)

• Record Type: Journal Article
• Authors: Yan, Tiantian; Shi, Jian; Li, Haojie; Luo, Zhongxuan; Wang, Zhihui
• Is Part Of: Pattern recognition. Volume 127(2022)
• Publication Date: 2022-07
• Subjects: Low-resolution -- Fine-grained image recognition -- Minimum spanning tree -- Semantic relation distillation; 00-01 -- 99-00; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108629
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: To the best of our knowledge, we are the first to address the issue of weakly supervised low-resolution fine-grained image recognition in an end-to-end manner. By enhancing the network's perception of discriminative features, the necessary critical details are recovered for fine-grained recognition, so as to improve the performance of weakly supervised low-resolution fine-grained image recognition. We propose a minimum spanning tree aggregation module to aggregate context information for each pixel by utilizing the structural characteristic of minimum spanning tree, which can help the fine-grained discriminative information restoration sub-network keep a watchful eye on discriminative fine-grained details. We introduce a semantic relation distillation loss to help the recognition sub-network calibrate the relationship between features, which further prompts the fine-grained detail restoration sub-network to generate the unambiguous details of super-resolution images and recognition sub-network to be aware of discriminative features. Extensive experiments are carried out on four challenging datasets (CUB-200-2011, Stanford Cars, FGVC-Aircraft and RP-281) to demonstrate the effectiveness of our framework. Abstract: The existing methods of fine-grained image recognition mainly devote to learning subtle yet discriminative features from the high-resolution input. However, their performance deteriorates significantly when they are used for low quality images because a lot of discriminative details of images are missing. We propose a discriminative information restoration and extraction network, termed as DRE-Net, to address the problem of low-resolution fine-grained image recognition, which has widespread application potential, such as shelf auditing and surveillance scenarios. DRE-Net is the first framework for weakly supervised low-resolution fine-grained image recognition and consists of two sub-networks: (1) fine-grained discriminative information restoration sub-network (FDR) and (2) recognition sub-network with the semantic relation distillation loss (SRD-loss). The first module utilizes the structural characteristic of minimum spanning tree (MST) to establish context information for each pixel by employing the spatial structures between each pixel and other pixels, which can help FDR focus on and restore the critical texture details. The second module employs the SRD-loss to calibrate recognition sub-network by transferring the correct relationships between every two pixels on the feature map. Meanwhile the SRD-loss can further prompt the FDR to recover reliable and accurate fine-grained details and guide the recognition sub-network to perceive the discriminative features from the correct relationships. Extensive experiments on three benchmark datasets and one retail product dataset demonstrate the effectiveness of our proposed framework.