Real-time automatic polyp detection in colonoscopy using feature enhancement module and spatiotemporal similarity correlation unit. (April 2021)

• Record Type: Journal Article
• Authors: Xu, Jianwei; Zhao, Ran; Yu, Yizhou; Zhang, Qingwei; Bian, Xianzhang; Wang, Jun; Ge, Zhizheng; Qian, Dahong
• Is Part Of: Biomedical signal processing and control. Volume 66(2021)
• Publication Date: 2021-04
• Subjects: Colonoscopy -- Automatic polyp detection -- Convolutional neural networks -- False positive relearning -- Image style transfer -- Feature enhancement -- Spatiotemporal information; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.102503
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: We combine the 2D CNN detector and spatiotemporal information to detect polyps in real time. We propose two feature enhancement modules to improve the performance of the CNN detector. We further integrate temporal and spatial information and improve performance in video detection. We have done sufficient validation experiments on both private and publicly available databases. The proposed method outperforms the existing methods which can meet real-time constraints. Abstract: Automatic detection of polyps is challenging because different polyps vary greatly, while the changes between polyps and their analogues are small. The state-of-the-art methods are based on convolutional neural networks (CNNs). However, they may fail due to lack of training data, resulting in high rates of missed detection and false positives (FPs). In order to solve these problems, our method combines the two-dimensional (2-D) CNN-based real-time object detector network with spatiotemporal information. Firstly, we use a 2-D detector network to detect static images and frames, and based on the detector network, we propose two feature enhancement modules—the FP Relearning Module (FPRM) to make the detector network learning more about the features of FPs for higher precision, and the Image Style Transfer Module (ISTM) to enhance the features of polyps for sensitivity improvement. In video detection, we integrate spatiotemporal information, which uses Structural Similarity (SSIM) to measure the similarity between video frames. Finally, we propose the Inter-frame Similarity Correlation Unit (ISCU) to combine the results obtained by the detector network and frame similarity to make the final decision. We verify our method on both private databases and publicly available databases. Experimental results show that these modules and units provide a performance improvement compared with the baseline method. Comparison with the state-of-the-art methods shows that the proposed method outperforms the existing ones which can meet real-time constraints. It's demonstrated that our method provides a performance improvement in sensitivity, precision and specificity, and has great potential to be applied in clinical colonoscopy.