Visual–inertial fusion-based registration between real and synthetic images in airborne combined vision system. (24th April 2019)

• Record Type: Journal Article
• Authors: Zhang, Lei; Zhai, Zhengjun; Niu, Wensheng; Wen, Pengcheng; He, Lang
• Is Part Of: International journal of advanced robotic systems. Volume 16:Number 2(2019:Mar./Apr.)
• Publication Date: 2019-04-24
• Subjects: Combined vision system -- conformal display -- runway detection -- visual–inertial fusion -- GPS-denied -- low visibility; Robotics -- Periodicals; Robotics; Periodicals; 629.892
• DOI: 10.1177/1729881419845528
• Languages: English
• ISSNs: 1729-8806

Abstract

Combined vision system is a perspective display concept to enhance a situation awareness of the pilots during aircraft landing, which integrates a real 2-D image captured from forward-looking infrared camera with a synthetic 3-D image derived by the aircraft pose and terrain database. However, the inertial measured errors significantly affect the conformal display of combined vision. This article proposes a novel method for real and synthetic images registration based on visual–inertial fusion. It includes the following key steps: (1) detect and extract the real runway features from forward-looking infrared image; (2) generate the synthetic runway features simultaneously; (3) set up vision measurement model with real and synthetic runway features; (4) integrate inertial data and visual observations in the square-root unscented Kalman filter; (5) create a synthetic 3-D scene by the filtered pose data and integrate it with a real 2-D image. The experimental results demonstrate that our method can guarantee the conformal display of combined vision system in GPS-denied and low visibility conditions.