Non-rigid object tracking via deep multi-scale spatial-temporal discriminative saliency maps. (April 2020) - Zhang, Pingping; Liu, Wei; Wang, Dong; Lei, Yinjie; Wang, Hongyu; Lu, Huchuan
Highlights: A novel effective non-rigid object tracking framework is proposed with the spatial-temporal consistent saliency detection. An efficient TFCN is developed to produce the local saliency prior for a given image region. A multi-scale multi-region mechanism is presented to generate multiple local saliency maps and then fuse them through a weighted entropy method. Extensive experiments on public saliency detection and visual tracking datasets show that our algorithm achieves considerably impressive results in both research fields. Abstract: In this paper, we propose a novel effective non-rigid object tracking framework based on the spatial-temporal consistent saliency detection. In contrast to most existing trackers that utilize a bounding box to specify the tracked target, the proposed framework can extract accurate regions of the target as tracking outputs. It achieves a better description of the non-rigid objects and reduces the background pollution for the tracking model. Furthermore, our model has several unique characteristics. First, a tailored fully convolutional neural network (TFCN) is developed to model the local saliency prior for a given image region, which not only provides the pixel-wise outputs but also integrates the semantic information. Second, a novel multi-scale multi-region mechanism is proposed to generate local saliency maps that effectively consider visual perceptions with different spatial layouts and scale variations. Subsequently, the local saliency maps are fused via a weighted entropy method, resulting in a discriminative saliency map. Finally, we present a non-rigid object tracking algorithm based on the predicted saliency maps. By utilizing a spatial-temporal consistent saliency map (STCSM), we conduct the target-background classification and use an online fine-tuning scheme for model updating. Extensive experiments demonstrate that the proposed algorithm achieves competitive performance in both saliency detection and visual tracking, especially outperforming other related trackers on the non-rigid object tracking datasets. Source codes and compared results are released at https://github.com/Pchank/TFCNTracker .