MSENet: A multi-scale enhanced network based on unique features guidance for medical image fusion. (April 2022)

• Record Type: Journal Article
• Authors: Li, Weisheng; Li, Ruyue; Fu, Jun; Peng, Xiuxiu
• Is Part Of: Biomedical signal processing and control. Volume 74(2022)
• Publication Date: 2022-04
• Subjects: Medical image fusion -- Multi-scale -- Unique feature fusion -- Deep learning -- Three-path dilated network; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.103534
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: The proposed MSENet model for multi-modal medical image fusion utilizes a densely three-path dilated network encoder to extract multi-scale features by enlarging the receptive field. The fusion process uses unique features from the source image to fuse the two images, preserving and enhancing unique and salient information. A skip connection is added in fusion image reconstruction to compensate for feature loss during encoding and obtain more comprehensive features. Abstract: Several medical image fusion methods based on deep learning have been proposed, but it remains difficult to design appropriate fusion rules. Single-scale networks also have problems with insufficient feature extraction. Therefore, this paper proposes MSENet, a multi-scale enhanced medical image fusion network based on unique feature guidance. To fully extract and fuse source image salient and unique features, first the proposed three-path dilated network includes different size filters in the encoder to extract multi-scale features. Second, the proposed unique feature fusion module only allows unique features to be fused and enhanced without designing additional fusion rules. Finally, the decoder comprises six convolution layers with one skip connection to reconstruct the final fused image with more detailed information from the source images. The proposed MSENet uses the "higher score is better" scoring strategy. We experimentally compared eight state-of-the-art methods on three multi-modal medical image datasets, with MSENet scores 8, 6 and 5 points higher than for the second optimal method, respectively. MSENet also provided more visually realistic fused images with rich textural details compared with the comparison methods.