Knowledge distillation approach towards melanoma detection. (July 2022)

• Record Type: Journal Article
• Authors: Khan, Md Shakib; Alam, Kazi Nabiul; Dhruba, Abdur Rab; Zunair, Hasib; Mohammed, Nabeel
• Is Part Of: Computers in biology and medicine. Volume 146(2022)
• Publication Date: 2022-07
• Subjects: Melanoma detection -- Knowledge distillation -- Skin lesion analysis -- Deep learning; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2022.105581
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Melanoma is regarded as the most threatening among all skin cancers. There is a pressing need to build systems which can aid in the early detection of melanoma and enable timely treatment to patients. Recent methods are geared towards machine learning based systems where the task is posed as image recognition, tag dermoscopic images of skin lesions as melanoma or non-melanoma. Even though these methods show promising results in terms of accuracy, they are computationally quite expensive to train, that questions the ability of these models to be deployable in a clinical setting or memory constraint devices. To address this issue, we focus on building simple and performant models having few layers, less than ten compared to hundreds. As well as with fewer learnable parameters, 0.26 million (M) compared to 42.5 M using knowledge distillation with the goal to detect melanoma from dermoscopic images. First, we train a teacher model using a ResNet-50 to detect melanoma. Using the teacher model, we train the student model known as Distilled Student Network (DSNet) which has around 0.26 M parameters using knowledge distillation achieving an accuracy of 91.7%. We compare against ImageNet pre-trained models such MobileNet, VGG-16, Inception-V3, EfficientNet-B0, ResNet-50 and ResNet-101. We find that our approach works well in terms of inference runtime compared to other pre-trained models, 2.57 s compared to 14.55 s. We find that DSNet (0.26 M parameters), which is 15 times smaller, consistently performs better than EfficientNet-B0 (4 M parameters) in both melanoma and non-melanoma detection across Precision, Recall and F1 scores. Highlights: Knowledge Distillation has been applied in dermoscopic images for the very first time. This approach aims to reduce complexity and facilitate deployment in edge devices. Self built 'DSNet' is being proposed as Student Model. This proposed method requires relatively less inference time to detect melanoma.