Multimodal super-resolution reconstruction of infrared and visible images via deep learning. (September 2022)

• Record Type: Journal Article
• Authors: Wang, Bowen; Zou, Yan; Zhang, Linfei; Li, Yuhai; Chen, Qian; Zuo, Chao
• Is Part Of: Optics and lasers in engineering. Volume 156(2022)
• Publication Date: 2022-09
• Subjects: Super-resolution -- Infrared image -- Convolutional neural network -- Multi-modal imaging -- Image fusion; Lasers in engineering -- Periodicals; Optical measurements -- Periodicals; Optics -- Periodicals; Lasers en ingénierie -- Périodiques; Mesures optiques -- Périodiques; Optique -- Périodiques; 621.36605
• DOI: 10.1016/j.optlaseng.2022.107078
• Languages: English
• ISSNs: 0143-8166

Abstract

Highlights: A deep-learning-based infrared-visible images fusion method based on encoder-decoder architecture is proposed. The image fusion task is reformulated as a problem of maintaining the structure and intensity ratio of the infrared-visible image. The proposed method can provide diverse imaging modalities and achieve superior performance in visual effects and objective assessments. It can stably provide high-resolution reconstruction results consistent with human visual observation. Abstract: In this paper, we propose a deep-learning-based infrared-visible images fusion method based on encoder-decoder architecture. The image fusion task is reformulated as a problem of maintaining the structure and intensity ratio of the infrared-visible image. The corresponding loss function is designed to expand the weight difference between the thermal target and the background. In addition, a single image super-resolution reconstruction based on a regression network is introduced to address the issue that traditional network mapping functions are not suitable for natural scenes. The forward generation and reverse regression models are considered to reduce the irrelevant function mapping space and approach the ideal scene data through double mapping constraints. Compared with other state-of-the-art approaches, our experimental results achieve superior performance in terms of both visual effects and objective assessments. In addition, it can stably provide high-resolution reconstruction results consistent with human visual observation while bridging the resolution gap between the infrared-visible images.