Fusion of thermal and visible cameras for the application of pedestrian detection. Issue 3 (March 2017)

• Record Type: Journal Article
• Authors: John, Vijay; Tsuchizawa, Shogo; Liu, Zheng; Mita, Seiichi
• Is Part Of: Signal, image and video processing. Volume 11:Issue 3(2017)
• Page Start: 517
• Page End: 524
• Publication Date: 2017-03
• Subjects: Sensor fusion -- Thermal imaging -- Optimization -- Pedestrian detection; Signal processing -- Digital techniques -- Periodicals; Image processing -- Digital techniques -- Periodicals; Digital video -- Periodicals; 621.3822
• DOI: 10.1007/s11760-016-0989-z
• Languages: English
• ISSNs: 1863-1703

Abstract

Abstract In this paper, we propose methods to calibrate visible and thermal cameras and register their images in the application of pedestrian detection. We calibrate the camera using a checkerboard pattern mounted on a heated rig. We implement the image registration using three different approaches. In the first approach, we use the camera calibration information to generate control points from the checkerboard pattern. These control points are then used to register the images. In the second approach, we generate trajectory points for image registration using an external illuminated object. In the third approach, we achieve the registration through face tracking without the aid of any external object. The particle swarm optimization algorithm performs the image registration using the generated control and trajectory points, observed in both the cameras. We demonstrate the advantages of fusing the thermal and visible camera within a pedestrian detection algorithm. We evaluate the proposed registration algorithms and perform a comparison with baseline algorithms, i.e. genetic and simulated annealing algorithms. Additionally, we also perform a detailed parameter evaluation of the particle swarm optimization algorithm. The experimental results demonstrate the accuracy of the proposed algorithm and the advantages of thermal-visible camera fusion.