Flow driven attention network for video salient object detection. Issue 6 (5th April 2020)

• Record Type: Journal Article
• Authors: Zhou, Feng; Shuai, Hui; Liu, Qingshan; Guo, Guodong
• Is Part Of: IET image processing. Volume 14:Issue 6(2020)
• Page Start: 997
• Page End: 1004
• Publication Date: 2020-04-05
• Subjects: image motion analysis -- video signal processing -- neural nets -- object detection -- image segmentation -- image sequences -- feature extraction; attention mechanism -- fusing optical flow -- appearance features -- feature map -- state‐of‐the‐art method flow‐guided recurrent neural encoder -- Densely Annotated Video Segmentation -- Freiburg–Berkeley Motion Segmentation -- flow driven attention network -- video salient object detection -- convolutional neural network -- still‐image based saliency detectors -- flow‐driven attention network -- FDAN -- motion information -- appearance feature extractor -- motion‐guided attention module -- saliency map regression module -- infers -- ultimate saliency map; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2019.0836
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : Salient object detection has been revolutionised by convolutional neural network (CNN) recently. However, it is hard to transfer the state‐of‐the‐art still‐image based saliency detectors to videos directly, owing to the neglect of temporal contexts between frames. In this study, the authors propose a flow‐driven attention network (FDAN) to exploit motion information for video salient object detection. FDAN consists of an appearance feature extractor, a motion‐guided attention module and a saliency map regression module. It extracts the appearance feature per frame, refines appearance feature with optical flow and infers the ultimate saliency map, respectively. Motion‐guided attention module is the core of FDAN, which extracts motion information in the form of attention. This attention mechanism is a two‐branch CNN, fusing optical flow and appearance features. In addition, a shortcut connection is applied to the attention multiplied feature map for noise suppression intensively. Experimental results show that the proposed method can achieve performance on par with the state‐of‐the‐art method flow‐guided recurrent neural encoder on challenging benchmarks of Densely Annotated Video Segmentation and Freiburg–Berkeley Motion Segmentation while being two times faster in detection.