Tracking more than 100 arbitrary objects at 25 FPS through deep learning. (January 2022)

• Record Type: Journal Article
• Authors: Vaquero, Lorenzo; Brea, Víctor M.; Mucientes, Manuel
• Is Part Of: Pattern recognition. Volume 121(2022)
• Publication Date: 2022-01
• Subjects: Multiple visual object tracking -- Motion estimation -- Deep learning -- Siamese networks; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108205
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A real-time multiple visual object tracker (MVOT) for motion estimation is proposed. Design of the first RoI operator able to work with backbones without padding. Definition of a novel pairwise cross-correlation operator for identity matching. Quality of our method is superior to is predecessor but with a 10-fold speedup. Abstract: Most video analytics applications rely on object detectors to localize objects in frames. However, when real-time is a requirement, running the detector at all the frames is usually not possible. This is somewhat circumvented by instantiating visual object trackers between detector calls, but this does not scale with the number of objects. To tackle this problem, we present SiamMT, a new deep learning multiple visual object tracking solution that applies single-object tracking principles to multiple arbitrary objects in real-time. To achieve this, SiamMT reuses feature computations, implements a novel crop-and-resize operator, and defines a new and efficient pairwise similarity operator. SiamMT naturally scales up to several dozens of targets, reaching 25 fps with 122 simultaneous objects for VGA videos, or up to 100 simultaneous objects in HD720 video. SiamMT has been validated on five large real-time benchmarks, achieving leading performance against current state-of-the-art trackers.