PCT: Pyramid convolutional transformer for parotid gland tumor segmentation in ultrasound images. (March 2023)

• Record Type: Journal Article
• Authors: Zhang, Gang; Zheng, Chenhong; He, Jianfeng; Yi, Sanli
• Is Part Of: Biomedical signal processing and control. Volume 81(2023)
• Publication Date: 2023-03
• Subjects: Transformer -- Attention mechanism -- Parotid gland tumor -- Medical image segmentation -- Dense pixel prediction -- Convolutional neural network; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2022.104498
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: We propose a Pyramid Convolutional Transformer (PCT) architecture based on the shrinking pyramid framework and Fusion Attention Transformer CNN (FTC) block for parotid gland tumors segmentation. In this architecture, the shrinking pyramid framework can effectively capture parotid gland tumor image features with dense pixel by integrating multi-scale dependencies of images. And the FTC block is constructed to address complex and variable contour characteristics of parotid gland tumors, which combines Transformer with CNN for preferable extracting global and local features of images by dual branch structure. Furthermore, in order to strengthen the connections of sequence information, we improve the multi-head attention mechanism (MHSA) as multi-head fusion attention (MHFA) module and introduce it into the Transformer branch. Finally, we establish PCT architecture based on FTC blocks and pyramid framework for accurate segmentation of parotid gland tumor images. Abstract: Preoperative segmentation of parotid gland tumor regions using deep learning is of great significance for treatment decisions. However, there are still two major limitations: to the best of our knowledge, no networks are designed specifically for parotid gland tumor segmentation tasks; and neither convolutional neural network (CNN) nor Transformer can extract both global and local feature solely. To address these issues, we first propose a Pyramid Convolutional Transformer (PCT) architecture based on the shrinking pyramid framework and Fusion Attention Transformer CNN (FTC) block for parotid gland tumors segmentation. In this architecture, the shrinking pyramid framework can effectively capture parotid gland tumor image features with dense pixel by integrating multi-scale dependencies of images. And the FTC block is constructed to address complex and variable contour characteristics of parotid gland tumors, which combines Transformer with CNN for preferable extracting global and local features of images by dual branch structure. The experimental results suggest that proposed PCT achieved intersection-over-union (IoU) of 0.8434 and Dice similarity coefficient (Dice) of 0.9151 on parotid gland tumor segmentation (PGTSeg) dataset, and attained new state-of-the-art performance on multiple challenging benchmarks with IoU of 0.8521 on MoNuSeg and IoU of 0.9080 on ISIC 2018. Meanwhile, common backbones equipped with FTC block outperformed the baseline model. The code and models will be available at: https://github.com/Twoverz/PCT-Pyramid-Convolutional-Transformer.