End‐to‐end learning interpolation for object tracking in low frame‐rate video. Issue 6 (1st April 2020)

• Record Type: Journal Article
• Authors: Liu, Liqiang; Cao, Jianzhong
• Is Part Of: IET image processing. Volume 14:Issue 6(2020)
• Page Start: 1066
• Page End: 1072
• Publication Date: 2020-04-01
• Subjects: video signal processing -- learning (artificial intelligence) -- object tracking -- interpolation -- mobile computing; low frame rates -- implicit video frame interpolation sub‐network -- object tracking -- low frame‐rate video -- high frame‐rate latent video -- effective end‐to‐end optimisation -- frame rate -- tracking accuracy -- semantic video analytics -- end‐to‐end learning interpolation -- subsequent semantic analytics -- bandwidth constraints -- analytics performance; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2019.0944
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : In many scenarios, where videos are transmitted through bandwidth‐limited channels for subsequent semantic analytics, the choice of frame rates has to balance between bandwidth constraints and analytics performance. Faced with this practical challenge, this study focuses on enhancing object tracking at low frame rates and proposes a learning Interpolation for tracking framework. This framework embeds an implicit video frame interpolation sub‐network, which is concatenated and jointly trained with another object tracking sub‐network. Once a low frame‐rate video is an input, it is first mapped into a high frame‐rate latent video, based on which the tracker is learned. Novel strategies and loss functions are derived to ensure the effective end‐to‐end optimisation of the authors' network. On several challenging benchmarks and settings, their method achieves a highly competitive tradeoff between frame rate and tracking accuracy. As is known, the implications of interpolation on semantic video analytics and tracking remain unexplored, and the authors expect their method to find many applications in mobile embedded vision, Internet of Things and edge computing.