Fusing disparate object signatures for salient object detection in video. (December 2017)

• Record Type: Journal Article
• Authors: Tu, Zhigang; Guo, Zuwei; Xie, Wei; Yan, Mengjia; Veltkamp, Remco C.; Li, Baoxin; Yuan, Junsong
• Is Part Of: Pattern recognition. Volume 72(2017:Dec.)
• Page Start: 285
• Page End: 299
• Publication Date: 2017-12
• Subjects: Spatiotemporal saliency computation -- Salient video object detection -- Object signatures -- Fusion; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.07.028
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We employ object signatures from complementary channels to help each other in improving salient object detection in respective channels in a video. A learning-based method combines various appearance and motion cues is introduced to predict motion boundaries, which are stable for foreground object detection in complex scenes. Foreground weights that are computed according to the identified object signatures are used to estimate saliency maps. A fusion method, which depends on the saliency information and object signatures, is proposed to integrate saliency maps from different channels to form a higher-quality spatiotemporal saliency maps. Abstract: We present a novel spatiotemporal saliency model for object detection in videos. In contrast to previous methods focusing on exploiting or incorporating different saliency cues, the proposed method aims to use object signatures which can be identified by any kinds of object segmentation methods. We integrate two distinctive saliency maps, which are respectively computed from object proposals of an appearance-dominated method and a motion-dominated algorithm, to obtain a refined spatiotemporal saliency maps. This enables the method to achieve good robustness and precision in identifying salient objects in videos under various challenging conditions. First, an improved appearance-based and a modified motion-based segmentation approaches are separately utilized to extract two kinds of candidate foreground objects. Second, with these captured object signatures, we design a new approach to filter the extracted noisy object pixels and label foreground superpixels in each object signature channel. Third, we introduce a foreground connectivity saliency measure to compute two types of saliency maps, from which an adaptive fusion strategy is exploited to obtain the final spatiotemporal saliency maps for salient object detection in a video. Both quantitative and qualitative experiments on several challenging video benchmarks demonstrate that the proposed method outperforms existing state-of-the-art approaches.