Non-rigid object tracking via deep multi-scale spatial-temporal discriminative saliency maps. (April 2020)

• Record Type: Journal Article
• Authors: Zhang, Pingping; Liu, Wei; Wang, Dong; Lei, Yinjie; Wang, Hongyu; Lu, Huchuan
• Is Part Of: Pattern recognition. Volume 100(2020:Apr.)
• Publication Date: 2020-04
• Subjects: Deep neural network -- Non-rigid object tracking -- Salient object detection -- Spatial-temporal consistency; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107130
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel effective non-rigid object tracking framework is proposed with the spatial-temporal consistent saliency detection. An efficient TFCN is developed to produce the local saliency prior for a given image region. A multi-scale multi-region mechanism is presented to generate multiple local saliency maps and then fuse them through a weighted entropy method. Extensive experiments on public saliency detection and visual tracking datasets show that our algorithm achieves considerably impressive results in both research fields. Abstract: In this paper, we propose a novel effective non-rigid object tracking framework based on the spatial-temporal consistent saliency detection. In contrast to most existing trackers that utilize a bounding box to specify the tracked target, the proposed framework can extract accurate regions of the target as tracking outputs. It achieves a better description of the non-rigid objects and reduces the background pollution for the tracking model. Furthermore, our model has several unique characteristics. First, a tailored fully convolutional neural network (TFCN) is developed to model the local saliency prior for a given image region, which not only provides the pixel-wise outputs but also integrates the semantic information. Second, a novel multi-scale multi-region mechanism is proposed to generate local saliency maps that effectively consider visual perceptions with different spatial layouts and scale variations. Subsequently, the local saliency maps are fused via a weighted entropy method, resulting in a discriminative saliency map. Finally, we present a non-rigid object tracking algorithm based on the predicted saliency maps. By utilizing a spatial-temporal consistent saliency map (STCSM), we conduct the target-background classification and use an online fine-tuning scheme for model updating. Extensive experiments demonstrate that the proposed algorithm achieves competitive performance in both saliency detection and visual tracking, especially outperforming other related trackers on the non-rigid object tracking datasets. Source codes and compared results are released at https://github.com/Pchank/TFCNTracker .