CASF-Net: Cross-attention and cross-scale fusion network for medical image segmentation. (February 2023)

• Record Type: Journal Article
• Authors: Zheng, Jianwei; Liu, Hao; Feng, Yuchao; Xu, Jinshan; Zhao, Liang
• Is Part Of: Computer methods and programs in biomedicine. Volume 229(2023)
• Publication Date: 2023-02
• Subjects: Biomedical image segmentation -- Medical images -- Dual-stream cross fusion -- Cross-attention mechanism -- Cross-scale feature fusion module; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.107307
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: We propose cross-attention and cross-scale fusion network (CASF-Net), which interacts the global and local features to maximize the joint advantages of CNN and Transformer for feature sublimation. The dual branch cross-fusion module (DCF) is proposed to efficiently aggregate the complementary information in different views of the same image, where the cross-attention mechanism aims to better integrate the information and meanwhile reduce the computational complexity. We utilize a cross-scale feature fusion module in the dual-branch decoder to enhance the contextual information of downstream tasks, by which the efficiency can also be enhanced. Extensive experiments on three typical tasks in medical image segmentation indicate that CASF-Net consistently outperforms other state-of-the-art methods, especially in the polyp segmentation task, validating the effectiveness of our model. The source code has been uploaded at https://github.com/ZhengJianwei2/CASF-Net . Abstract: Background: Automatic segmentation of medical images has progressed greatly owing to the development of convolutional neural networks (CNNs). However, there are two uncertainties with current approaches based on convolutional operations: (1) how to eliminate the general limitations that CNNs lack the ability of modeling long-range dependencies and global contextual interactions, and (2) how to efficiently discover and integrate global and local features that are implied in the image. Notably, these two problems are interconnected, yet previous approaches mainly focus on the first problem and ignore the importance of information integration. Methods: In this paper, we propose a novel cross-attention and cross-scale fusion network (CASF-Net), which aims to explicitly tap the potential of dual-branch networks and fully integrate the coarse and fine-grained feature representations. Specifically, the well-designed dual-branch encoder hammers at modeling non-local dependencies and multi-scale contexts, significantly improving the quality of semantic segmentation. Moreover, the proposed cross-attention and cross-scale module efficiently perform multi-scale information fusion, being capable of further exploring the long-range contextual information. Results: Extensive experiments conducted on three different types of medical image segmentation tasks demonstrate the state-of-the-art performance of our proposed method both visually and numerically. Conclusions: This paper assembles the feature representation capabilities of CNN and transformer and proposes cross-attention and cross-scale fusion algorithms. The promising results show new possibilities of using cross-fusion mechanisms in more downstream medical image tasks.