Deep gated attention networks for large-scale street-level scene segmentation. (April 2019)

• Record Type: Journal Article
• Authors: Zhang, Pingping; Liu, Wei; Wang, Hongyu; Lei, Yinjie; Lu, Huchuan
• Is Part Of: Pattern recognition. Volume 88(2019:Apr.)
• Page Start: 702
• Page End: 714
• Publication Date: 2019-04
• Subjects: Scene segmentation -- Fully convolutional network -- Spatial gated attention -- Street-level image understanding; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2018.12.021
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel spatial gated attention mechanism is proposed in the context of pixel-wise labeling tasks. A multi-scale feature interaction mechanism is proposed for hierarchical feature aggregation. Different levels of features are re-weighted with the local structure and contextual information. The proposed GANet can better capture the scene layout and multi-level information of street-view images. State-of-the-art performance on three challenging large-scale scene segmentation benchmarks is achieved. Abstract: Street-level scene segmentation aims to label each pixel of street-view images into specific semantic categories. It has been attracting growing interest due to various real-world applications, especially in the area of autonomous driving. However, this pixel-wise labeling task is very challenging under the complex street-level scenes and large-scale object categories. Motivated by the scene layout of street-view images, in this work we propose a novel Spatial Gated Attention (SGA) module, which automatically highlights the attentive regions for pixel-wise labeling, resulting in effective street-level scene segmentation. The proposed module takes as input the multi-scale feature maps based on a Fully Convolutional Network (FCN) backbone, and produces the corresponding attention mask for each feature map. The learned attention masks can neatly highlight the regions of interest while suppress background clutter. Furthermore, we propose an efficient multi-scale feature interaction mechanism which is able to adaptively aggregate the hierarchical features. Based on the proposed mechanism, the features of different levels are adaptively re-weighted according to the local spatial structure and the surrounding contextual information. Consequently, the proposed modules are able to boost standard FCN architectures and result in an enhanced pixel-wise segmentation for street-level scene images. Extensive experiments on three public available street-level benchmarks demonstrate that the proposed Gated Attention Network (GANet) approach achieves consistently superior performance and outperforms the very recent state-of-the-art methods.