Deep convolutional neural network for classification of thyroid nodules on ultrasound: Comparison of the diagnostic performance with that of radiologists. Issue 152 (July 2022)

• Record Type: Journal Article
• Authors: Kim, Yeon-Jae; Choi, Yangsean; Hur, Su-Jin; Park, Ki-Sun; Kim, Hyun-Jin; Seo, Minkook; Lee, Min Kyoung; Jung, So-Lyung; Jung, Chan Kwon
• Is Part Of: European journal of radiology. Issue 152(2022)
• Publication Date: 2022-07
• Subjects: Thyroid Nodule -- Deep Learning -- Biopsy, Fine-Needle -- Sensitivity and Specificity; DL deep learning -- FNAB fine-needle aspiration biopsy -- Grad-CAM gradient-weighted class activation mapping -- IoU intersection over union -- R-CNN region-based convolutional neural network -- ResNet deep residual learning for image recognition -- VGG very deep convolutional networks for large-scale image recognition; Medical radiology -- Periodicals; Radiology -- Periodicals; Radiologie médicale -- Périodiques; Medical radiology; Periodicals; 616.075705
• DOI: 10.1016/j.ejrad.2022.110335
• Languages: English
• ISSNs: 0720-048X

Abstract

Highlights: On ultrasound images, deep learning-trained models demonstrated comparable diagnostic performance to radiologists in differentiating malignant from benign thyroid nodules. VGG16 model showed the best diagnostic performance in internal (AUC, 0.86; sensitivity, 91.8%; specificity, 73.2%) and external (AUC: 0.83; sensitivity: 78.6%; specificity: 76.8%) test sets. Deep learning models may help radiologists' diagnosis of thyroid nodules on ultrasound. Abstract: Purpose: This study aimed to train and validate deep learning (DL) models for differentiating malignant from benign thyroid nodules on US images and compare their performance with that of radiologists. Methods: Images of thyroid nodules in patients who underwent US-guided fine-needle aspiration biopsy at our institution between January 2010 and March 2020 were retrospectively reviewed. Four radiologists independently classified the images. Images of thyroid nodules were trained using three different image classification DL models (VGG16, VGG19, and ResNet). The diagnostic performances of the DL models were calculated for the internal and external datasets and compared with the diagnoses of the four radiologists. Pairwise comparisons of the AUCs between the radiologists and DL models were made using bootstrap-based tests. Results: In total, 15, 409 images from 7, 321 patients (mean age, 60 ± 13 years; malignant nodules, 20.7%) were randomly grouped into training (n = 12, 327) and validation (n = 3, 082) sets. Independent internal (n = 432; 197 patients) and external (n = 168; 59 patients) test sets were also acquired. The DL models demonstrated a higher diagnostic performance than the radiologists in the internal test set (AUC, 0.83 – 0.86 vs. 0.71 – 0.76, P < 0.05), but not in the external test set. The VGG16 model demonstrated the highest diagnostic performance in internal (AUC, 0.86; sensitivity, 91.8%; specificity, 73.2%) and external (AUC: 0.83; sensitivity: 78.6%; specificity: 76.8%) test sets. However, no statistical differences were found in the AUCs among the DL models. Conclusions: The DL models demonstrated comparable diagnostic performance to radiologists in distinguishing benign from malignant thyroid nodules on US images and may play a potential role in augmenting radiologists' diagnosis of thyroid nodules.