Multi-modal medical image super-resolution fusion based on detail enhancement and weighted local energy deviation. (February 2023)

• Record Type: Journal Article
• Authors: Yang, Yong; Cao, Sihua; Wan, Weiguo; Huang, Shuying
• Is Part Of: Biomedical signal processing and control. Volume 80:Part 2(2023)
• Publication Date: 2023-02
• Subjects: Multi-modal medical image fusion -- Super-resolution -- Detail enhancement -- Weighted local energy deviation; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104387
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: A novel multimodal medical image fusion (MMIF) method is proposed based on detail enhancement and weighted local energy deviation. A detail layer fusion rule is proposed to effectively retain the detail information and salient information of the source images. A base layer fusion rule is designed to effectively retain the energy and structure information of the source images. In this work, medical image super-resolution technology is introduced into MMIF for the first time. Abstract: Multi-modal medical image fusion (MMIF) integrates medical images of different modalities into an image with rich information to boost the accuracy and efficiency of clinical diagnosis and treatment. There are two main problems in medical image fusion: 1) It is difficult to balance the computational efficiency and fusion quality; 2) in the clinic, it is necessary to observe medical images in high resolution. To overcome these problems, a multi-modal medical image super-resolution (SR) fusion (MMISRF) method is proposed based on detail enhancement and weighted local energy deviation (WLED). The method has two major novelties. For the first problem, to improve the efficiency, the proposed method decomposes the SR image into one base layer and one detail layer through a two-scale decomposition method. Then, for the detail layer, a fusion rule is proposed based on detail enhancement and information refinement to enhance the detail information and retain the salient feature information. For the base layer, a WLED-based rule is designed to better preserve the energy information from the source images to the fused image. The final fused image is obtained by combining the fused detail and base layers. For the second problem, this paper introduces the bicubic interpolation-based SR into the field of MMIF for the first time. The experimental results indicate that the proposed MMISRF outperforms the state-of-the-art approaches in terms of subjective visual effect and objective evaluation. Furthermore, the proposed method is more effective for MMIF task at different resolutions.