Sparse attention block: Aggregating contextual information for object detection. (April 2022)

• Record Type: Journal Article
• Authors: Chen, Chunlin; Yu, Jun; Ling, Qiang
• Is Part Of: Pattern recognition. Volume 124(2022)
• Publication Date: 2022-04
• Subjects: Object detection -- Self-attention -- Convolution neural network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108418
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: This paper proposes a sparse attention block (SA block) to capture long-range dependencies in an efficient way. SA block introduces the additional cost of < 2 % GPU memory and computation of the conventional non-local block. SA block can be easily plugged into various object detection frameworks. SA block boosts the detection accuracy by more than 1% on COCO with slight computation and memory addition. Abstract: It is well recognized that the contextual information of surrounding objects is beneficial for object detection. Such contextual information can often be obtained from long-range dependencies. This paper proposes a sparse attention block to capture long-range dependencies in an efficient way. Unlike the conventional non-local block, which generates a dense attention map to characterize the dependency between any two positions of the input feature map, our sparse attention block samples the most representative positions for contextual information aggregation. After searching for local peaks in a heat map of the given input feature map, it adaptively selects a sparse set of positions to represent the relationship between query and key elements. With the obtained sparse positions, our sparse attention block can well model long-range dependencies, and greatly improve the object detection performance at the additional cost of < 2% GPU memory and computation of the conventional non-local block. This sparse attention block can be easily plugged into various object detection frameworks, such as Faster R-CNN, RetinaNet and Mask R-CNN. Experiments on COCO benchmark confirm that our sparse attention block can boost the detection accuracy with significant gains ranging from 1.4% to 1.9% and negligible overhead of computation and memory usage.