AMFNet: An attention-guided generative adversarial network for multi-model image fusion. (September 2022)

• Record Type: Journal Article
• Authors: Wang, Jing; Yu, Long; Tian, Shengwei; Wu, Weidong; Zhang, Dezhi
• Is Part Of: Biomedical signal processing and control. Volume 78(2022)
• Publication Date: 2022-09
• Subjects: Image fusion -- Self-attention -- Generative adversarial network -- Convolutional block attention module; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.103990
• Languages: English
• ISSNs: 1746-8094

Abstract

Graphical abstract: Highlights: We propose an attention-guided generative adversarial network for multi-model image fusion to solve the problem of long-range dependencies and strengthen the ability of information extraction and feature representations. We introduce an attention mechanism into the discriminator. It can restrict the discriminator to focus on the detail information and the attention region, reduce excessive attention to the source image, so as to retain more detail feature information. The proposed AMFNet can be used not only for medical image fusion but also for infrared and visible image fusion. Abstract: Most of the existing image fusion methods fail to retain sufficient salient information, lack focuses on the most discriminative regions of the image, and often neglect the subjective perception of the human visual system. To address these problems, we propose an attention-guided generative adversarial network (AMFNet) for multi-model image fusion. The generator network of AMFNet consists of three parts: an attention network for capturing long-range dependencies in the internal representations of images, an information refinement network for obtaining image feature maps, and a fusion network for merging the attention network and the information refinement network. In addition, the convolutional block attention module is introduced to force the discriminator to focus on the most discriminative regions of the multi-modal source images. The results of qualitative and quantitative experiments conducted on numerous public datasets demonstrate that the proposed method outperforms other methods on visual effects and retains more detail information about the images.