Multi-state colposcopy image fusion for cervical precancerous lesion diagnosis using BF-CNN. (July 2021)

• Record Type: Journal Article
• Authors: Yan, Ling; Li, Shufeng; Guo, Yi; Ren, Peng; Song, Haoxuan; Yang, Jingjing; Shen, Xingfa
• Is Part Of: Biomedical signal processing and control. Volume 68(2021)
• Publication Date: 2021-07
• Subjects: Colposcopy -- Cervical cancer -- Cervicogram -- Convolutional neural network -- Feature fusion -- Factorized bilinear pooling; Signal processing -- Periodicals; Biomedical engineering -- Periodicals; Signal Processing, Computer-Assisted -- Periodicals; Image Processing, Computer-Assisted -- Periodicals; Biomedical Engineering -- Periodicals; 610.28
• DOI: 10.1016/j.bspc.2021.102700
• Languages: English
• ISSNs: 1746-8094

Abstract

Highlights: Introducing a novel convolutional network framework combining two-state images information for diagnosis of cervical dysplasia. Developing an attention-based feature selection module, which can aggregate the associated information from the feature maps of two-state images as a supplement to the classification information. The performance of our method outperforms that of existing single-state and multi-modal algorithms and manual diagnosis. Abstract: Colposcopy is one of the most commonly used procedures to screen for cervical lesions in early precancerous stages, allowing effective preventive treatments for cervical cancer. Deep learning based algorithms have been proven promising for analyzing colposcopy images (i.e., cervicograms) to identify cervical precancerous lesions. However, most existing methods only use the single-state images applied with acetic acid solution (i.e., acetic acid cervicograms) as their input data, while missing the additional information provided by colposcopy images applied with Lugol's iodine solution (i.e., Lugol's iodine cervicograms). Since both acetic acid and Lugol's iodine cervicograms are available in clinic, health providers would select suspected lesion regions for further biopsy by visually inspecting both types of cervicograms. Therefore, it is essential to take these two-state images as inputs to extract features because these two types of cervicograms provide complementary information. In this work, we propose a bilinear fuse convolutional neural network (BF-CNN) that implements a feature selection module based on attention mechanisms and utilizes the factorized bilinear pooling technique to effectively fuse two-state image features for automatic diagnosis of cervical precancerous lesions. We applied BF-CNN and alternative algorithms to the real clinic cervicogram dataset consisted of 1400 patients, where both types of cervicograms were available per patient. As a result, BF-CNN obtained similar sensitivity 74.6% as well as the highest accuracy 85.5%, specificity 95.7%, and AUC 0.909, compared with known multi-modal algorithms and our improved algorithm only using one type of cervicograms. BF-CNN is expected to provide a useful tool to help physician make more precise clinic diagnosis.