A multiscale double-branch residual attention network for anatomical–functional medical image fusion. (February 2022)

• Record Type: Journal Article
• Authors: Li, Weisheng; Peng, Xiuxiu; Fu, Jun; Wang, Guofen; Huang, Yuping; Chao, Feifei
• Is Part Of: Computers in biology and medicine. Volume 141(2022)
• Publication Date: 2022-02
• Subjects: Image fusion -- Multiscale -- Double branches -- Residual -- Attention; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2021.105005
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Medical image fusion technology synthesizes complementary information from multimodal medical images. This technology is playing an increasingly important role in clinical applications. In this paper, we propose a new convolutional neural network, which is called the multiscale double-branch residual attention (MSDRA) network, for fusing anatomical–functional medical images. Our network contains a feature extraction module, a feature fusion module and an image reconstruction module. In the feature extraction module, we use three identical MSDRA blocks in series to extract image features. The MSDRA block has two branches. The first branch uses a multiscale mechanism to extract features of different scales with three convolution kernels of different sizes, while the second branch uses six 3 × 3 convolutional kernels. In addition, we propose the Feature L1 -Norm fusion strategy to fuse the features obtained from the input images. Compared with the reference image fusion algorithms, MSDRA consumes less fusion time and achieves better results in visual quality and the objective metrics of Spatial Frequency ( SF ), Average Gradient ( AG ), Edge Intensity ( EI ), Quality-Aware Clustering ( QAC ), Variance ( VAR ), and Visual Information Fidelity for Fusion ( VIFF ). Highlights: A new convolutional neural network (MSDRA) is applied to extract image features. The Feature L1 -Norm fusion strategy is implemented in the fusion process. Our fusion results have better performance of objective metrics. Our fusion results provide clearer details in fusion images.