Perspective registration and multi-frame super-resolution of license plates in surveillance videos. (March 2021)

• Record Type: Journal Article
• Authors: Guarnieri, Gabriele; Fontani, Marco; Guzzi, Francesco; Carrato, Sergio; Jerian, Martino
• Is Part Of: Forensic science international. Volume 36(2021)
• Publication Date: 2021-03
• Subjects: Super-resolution -- Video surveillance -- Object tracking -- Image registration -- Perspective transformation
• DOI: 10.1016/j.fsidi.2020.301087
• Languages: English
• ISSNs: 2666-2817

Abstract

Abstract: One task often encountered in surveillance videos is the recognition of a target—e.g. the license plate of a vehicle. Often, the quality of a single video frame does not permit a reliable recognition. If multiple frames are available, it is possible to combine them in order to generate a single image with lower noise (frame averaging) and/or higher resolution (super-resolution). In order for these techniques to work, it is necessary to accurately estimate the motion of the object of interest in the recorded footage. In this paper, we introduce a method capable of accurately computing the perspective transformation that describes the motion of a planar object. The method minimizes the squared distance between the transformed image and a reference, computed over a user-defined region of interest, and uses the partial derivatives in order to significantly speed up the computation. This approach is inspired by the well known Kanade–Lucas–Tomasi feature tracker. Highlights: In this paper, we introduce a method capable of accurately computing the perspective transformation that describes the motion of a planar object. A fast tracking algorithm is provided for quadrilateral-shaped planar objects, the main application being of course vehicles license plates tracking. The registration algorithm is used in a custom fast multi-frame super-resolution framework. A realistic forensic-use dataset, generated and used for comprehensive testing of the entire system, is made available to the public.