Evaluating generic AutoML tools for computational pathology. (2022)

• Record Type: Journal Article
• Authors: Schwen, Lars Ole; Schacherer, Daniela; Geißler, Christian; Homeyer, André
• Is Part Of: Informatics in medicine unlocked. Volume 29(2022)
• Publication Date: 2022
• Subjects: Computational pathology -- Convolutional neural networks -- AutoML -- Tissue classification -- Hyperparameter optimization -- Reproducibility; Medical informatics -- Periodicals; 610.285
• DOI: 10.1016/j.imu.2022.100853
• Languages: English
• ISSNs: 2352-9148

Abstract

Abstract: Image analysis tasks in computational pathology are commonly solved using convolutional neural networks (CNNs). The selection of a suitable CNN architecture and hyperparameters is usually done through exploratory iterative optimization, which is computationally expensive and requires substantial manual work. The goal of this article is to evaluate how generic tools for neural network architecture search and hyperparameter optimization perform for common use cases in computational pathology. For this purpose, we evaluated one on-premises and one cloud-based tool for three different classification tasks for histological images: tissue classification, mutation prediction, and grading. We found that the default CNN architectures and parameterizations of the evaluated AutoML tools already yielded classification performance on par with the original publications. Hyperparameter optimization for these tasks did not substantially improve performance, despite the additional computational effort. However, performance varied substantially between classifiers obtained from individual AutoML runs due to non-deterministic effects. Generic CNN architectures and AutoML tools could thus be a viable alternative to manually optimizing CNN architectures and parametrizations. This would allow developers of software solutions for computational pathology to focus efforts on harder-to-automate tasks such as data curation. Highlights: We evaluated two generic AutoML tools for three computational pathology use cases. The performance of the resulting classifiers was on par with literature results. Generic presets were sufficient, hyperparameter optimization was of little benefit. Inference results varied substantially due to non-determinism in Deep Learning.