CEFusion: Multi‐Modal medical image fusion via cross encoder. Issue 12 (9th June 2022)

• Record Type: Journal Article
• Authors: Zhu, Ya; Wang, Xue; Chen, Luping; Nie, Rencan
• Is Part Of: IET image processing. Volume 16:Issue 12(2022)
• Page Start: 3177
• Page End: 3189
• Publication Date: 2022-06-09
• Subjects: Image processing -- Periodicals; 621.36705
• DOI: 10.1049/ipr2.12549
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract: Most existing deep learning‐based multi‐modal medical image fusion (MMIF) methods utilize single‐branch feature extraction strategies to achieve good fusion performance. However, for MMIF tasks, it is thought that this structure cuts off the internal connections between source images, resulting in information redundancy and degradation of fusion performance. To this end, this paper proposes a novel unsupervised network, termed CEFusion. Different from existing architecture, a cross‐encoder is designed by exploiting the complementary properties between the original image to refine source features through feature interaction and reuse. Furthermore, to force the network to learn complementary information between source images and generate the fused image with high contrast and rich textures, a hybrid loss is proposed consisting of weighted fidelity and gradient losses. Specifically, the weighted fidelity loss can not only force the fusion results to approximate the source images but also effectively preserve the luminance information of the source image through weight estimation, while the gradient loss preserves the texture information of the source image. Experimental results demonstrate the superiority of the method over the state‐of‐the‐art in terms of subjective visual effect and quantitative metrics in various datasets.