Testing the Ability of Convolutional Neural Networks to Learn Radiomic Features. (June 2022)

• Record Type: Journal Article
• Authors: Klyuzhin, Ivan S.; Xu, Yixi; Ortiz, Anthony; Ferres, Juan Lavista; Hamarneh, Ghassan; Rahmim, Arman
• Is Part Of: Computer methods and programs in biomedicine. Volume 219(2022)
• Publication Date: 2022-06
• Subjects: Deep learning -- Radiomics -- Cancer -- Medical imaging -- Image analysis; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.106750
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: We tested the ability of convolutional neural nets to effectively capture intensity, shape, and texture of tumors in medical images. Convolutional neural networks were found to have a limited capacity to capture tumor shape and heterogeneity. Shape and texture features required an order-of-magnitude more training samples compared to intensity features. Convolutional neural networks combined with explicit radiomics should yield the most powerful predictive models. Abstract: Background and objective: Radiomics and deep learning have emerged as two distinct approaches to medical image analysis. However, their relative expressive power remains largely unknown. Theoretically, hand-crafted radiomic features represent a mere subset of features that neural networks can approximate, thus making deep learning a more powerful approach. On the other hand, automated learning of hand-crafted features may require a prohibitively large number of training samples. Here we directly test the ability of convolutional neural networks (CNNs) to learn and predict the intensity, shape, and texture properties of tumors as defined by standardized radiomic features. Methods: Conventional 2D and 3D CNN architectures with an increasing number of convolutional layers were trained to predict the values of 16 standardized radiomic features from real and synthetic PET images of tumors, and tested. In addition, several ImageNet-pretrained advanced networks were tested. A total of 4000 images were used for training, 500 for validation, and 500 for testing. Results: Features quantifying size and intensity were predicted with high accuracy, while shape irregularity and heterogeneity features had very high prediction errors and generalized poorly. For example, mean normalized prediction error of tumor diameter with a 5-layer CNN was 4.23 ± 0.25, while the error for tumor sphericity was 15.64 ± 0.93. We additionally found that learning shape features required an order of magnitude more samples compared to intensity and size features. Conclusions: Our findings imply that CNNs trained to perform various image-based clinical tasks may generally under-utilize the shape and texture information that is more easily captured by radiomics. We speculate that to improve the CNN performance, shape and texture features can be computed explicitly and added as auxiliary variables to the networks, or supplied as synthetic inputs.