Learning sequentially diversified representations for fine-grained categorization. (January 2022)

• Record Type: Journal Article
• Authors: Zhang, Lianbo; Huang, Shaoli; Liu, Wei
• Is Part Of: Pattern recognition. Volume 121(2022)
• Publication Date: 2022-01
• Subjects: Fine-grained visual categorization -- Convolutional neural networks -- Diversity learning -- Object recognition; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108219
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We present Sequentially Diversified Networks (SDNs) for fine-grained visual categorization. SDNs is composed of multiple lightweight sub-networks to learn different scales of discriminative regions. On top of one shared backbone, this design avoids multiple backbones or forward passes thus maintaining efficiency. We introduce a diversified constraint function that explicitly promotes feature diversity among branches while preserving class discrimination. SDNs reports state-of-the-art performance on three challenging datasets, including CUB-200-2011, Stanford Cars and FGVC-Aircraft. Abstract: Learning representation carrying rich local information is essential for recognizing fine-grained objects. Existing methods to this task resort to multi-stage frameworks to capture fine-grained information. However, they usually require multiple forward passes of the backbone network, resulting in efficiency deterioration. In this paper, we propose Sequentially Diversified Networks (SDNs) that enrich representation by promoting their diversity while maintaining the extraction efficiency. Specifically, we construct multiple lightweight sub-networks to model mutually different scales of discriminative patterns. The design of these sub-networks follows the sequentially diversified constraint, encouraging them to be varied in spatial attention. By inserting these sub-networks into a single backbone network, SDNs enable information interaction among local regions of the fine-grained image. In this way, SDNs jointly promote diversity in terms of scale and spatial attention in the one-stage pipeline, thereby facilitating the learning of diversified representation efficiently. We evaluate our proposed method on three challenging datasets, namely CUB-200-2011, Stanford-Cars, and FGVC-Aircraft. Experiments demonstrate its effectiveness in learning diversified information. Moreover, our method achieves state-of-the-art performance, only requiring a single forward pass of the backbone network, which reduces inference time noticeably.