Three-step action search networks with deep Q-learning for real-time object tracking. (May 2020)

• Record Type: Journal Article
• Authors: Teng, Zhu; Zhang, Baopeng; Fan, Jianping
• Is Part Of: Pattern recognition. Volume 101(2020:May)
• Publication Date: 2020-05
• Subjects: Object tracking -- Deep Q-learning -- Action search network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107188
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A Three-Step Action Search network (TSAS) is designed for real-time object tracking. A collaborative learning strategy is developed to learn the TSAS network in order to achieve more discriminative. TSAS network is supervised in a sense of action classification and is also formulated as a reinforcement learning of cumulative rewards along the action steps and the time steps. Two action-value functions are approximated through deep networks in order to determine the best action for object tracking. Deep reinforcement learning is exploited to deal with the localization delay in the action steps effectively and explore the long-term information in videos efficiently. Abstract: Sliding window and candidate sampling are two widely used search strategies for visual object tracking, but they are far behind real-time. By treating the tracking problem as a three-step decision-making process, a novel tracking network, which explores only three small subsets of candidate regions, is developed to achieve faster (real-time) localization of the target object along the frames in a video. A convolutional neural network agent is formulated to interact with a video over time, and two action-value functions are exploited to learn a favorable policy off-line to determine the best action for visual object tracking. Our model is trained in a collaborative learning way by using action classification and cumulative reward approximation in reinforcement learning. We have evaluated our proposed tracker against a number of state-of-the-art ones over three popular tracking benchmarks including OTB-2013, OTB-2015, and VOT2017. The experimental results have demonstrated that our proposed method can achieve very competitive performance on real-time object tracking.