Attention-shift based deep neural network for fine-grained visual categorization. (August 2021)

• Record Type: Journal Article
• Authors: Niu, Yi; Jiao, Yang; Shi, Guangming
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: Fine-grained visual categorization -- Deep neural network -- Human perception mechanism -- Attention-shift -- Encoder-decoder; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107947
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We re-investigate the pipeline of fine-grained visual categorization (FGVC) techniques from the view of human visual recognition system, and propose a novel Attention-Shift based Deep Neural Network (AS-DNN) for automatic parts locating and semantic correlation learning. We propose an end-to-end trainable sub-network structure C sft to simulate the attention-shift process. C sft locates the discriminative regions automatically and encodes and decodes the semantic relations among diverse discriminative parts iteratively. Comprehensive experiments show that AS-DNN achieves state-of-the-art performances in three widely used challenging datasets. Moreover, the visualization of located discriminative parts proves the robustness of AS-DNN in complex backgrounds and postures. Abstract: Fine-grained visual categorization (FGVC) has attracted extensive attention in recent years. The general pipeline of current FGVC techniques is to 1) locate the discriminative regions; 2) extract features from each region independently; and 3) feed the integrated features to a classifier. In this paper, we re-investigate the pipeline from the view of human visual recognition mechanisms. The perceiving of discriminative regions is a temporal processing by the human visual system (HVS) via the attention-shift mechanism. However, the existing independent feature extracting and one-pass feeding strategy ignore the inherent semantic relationships among discriminative regions, and thus is improper to model the attention-shift process properly. Therefore, in this paper, we propose a novel end-to-end FGVC network structure named Attention-Shift based Deep Neural Network (AS-DNN) to locate the discriminative regions automatically and encode the semantic correlations iteratively. AS-DNN consists of two channels: 1) the global perception channel C glb and 2) the attention-shift channel C sft, simulating the global perception and the attention-shift mechanism, respectively. Experimental results show that AS-DNN achieves state-of-the-art performances by outperforming both the CNN-based weakly or strongly-supervised FGVC algorithms on several widely-used fine-grained datasets, and the visualization of attention regions exhibit that the proposed method can locate the discriminative regions robustly in complex backgrounds and postures.