Tuning hyperparameters of machine learning algorithms and deep neural networks using metaheuristics: A bioinformatics study on biomedical and biological cases. (April 2022)

• Record Type: Journal Article
• Authors: Nematzadeh, Sajjad; Kiani, Farzad; Torkamanian-Afshar, Mahsa; Aydin, Nizamettin
• Is Part Of: Computational biology and chemistry. Volume 97(2022)
• Publication Date: 2022-04
• Subjects: Tuning -- Hyperparameters -- Machine learning -- Deep learning -- Metaheuristics -- Bioinformatics; Chemistry -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; Biochemistry -- Data processing; Biology -- Data processing; Molecular biology -- Data processing; Periodicals; Electronic journals; 542.85
• DOI: 10.1016/j.compbiolchem.2021.107619
• Languages: English
• ISSNs: 1476-9271

Abstract

Abstract: The performance of a model in machine learning problems highly depends on the dataset and training algorithms. Choosing the right training algorithm can change the tale of a model. While some algorithms have a great performance in some datasets, they may fall into trouble in other datasets. Moreover, by adjusting hyperparameters of an algorithm, which controls the training processes, the performance can be improved. This study contributes a method to tune hyperparameters of machine learning algorithms using Grey Wolf Optimization (GWO) and Genetic algorithm (GA) metaheuristics. Also, 11 different algorithms including Averaged Perceptron, FastTree, FastForest, Light Gradient Boost Machine (LGBM), Limited memory Broyden Fletcher Goldfarb Shanno algorithm Maximum Entropy (LbfgsMxEnt), Linear Support Vector Machine (LinearSVM), and a Deep Neural Network (DNN) including four architectures are employed on 11 datasets in different biological, biomedical, and nature categories such as molecular interactions, cancer, clinical diagnosis, behavior related predictions, RGB images of human skin, and X-rays images of Covid19 and cardiomegaly patients. Our results show that in all trials, the performance of the training phases is improved. Also, GWO demonstrates a better performance with a p-value of 2.6E-5. Moreover, in most experiment cases of this study, the metaheuristic methods demonstrate better performance and faster convergence than Exhaustive Grid Search (EGS). The proposed method just receives a dataset as an input and suggests the best-explored algorithm with related arguments. So, it is appropriate for datasets with unknown distribution, machine learning algorithms with complex behavior, or users who are not experts in analytical statistics and data science algorithms. Graphical Abstract: ga1 Highlights: The proposed method can improve the performance of the machine learning model independent from the composition of datasets. It assists to gain a better performance faster than tuning by blindly and randomly chosen hyperparameters. The idea of this method is general and it is not specific to a limited number of training algorithms and optimizers. This method is convenient to use for users who are not experts in algorithm designing, also for real-world problems. Generally, metaheuristic methods give better result and converge faster than blind approaches like Exhaustive Grid Search.