Simplified swarm optimization for hyperparameters of convolutional neural networks. (March 2023)

• Record Type: Journal Article
• Authors: Yeh, Wei-Chang; Lin, Yi-Ping; Liang, Yun-Chia; Lai, Chyh-Ming; Huang, Chia-Ling
• Is Part Of: Computers & industrial engineering. Volume 177(2023)
• Publication Date: 2023-03
• Subjects: Machine learning -- Image recognition -- Convolutional neural networks -- Simplified swarm optimization -- Hyperparameter optimization; Engineering -- Data processing -- Periodicals; Industrial engineering -- Periodicals; 620.00285
• DOI: 10.1016/j.cie.2023.109076
• Languages: English
• ISSNs: 0360-8352

Abstract

Highlights: A hyperparameter optimization algorithm called SSO-LeNet is proposed to improve LeNet. SSO-LeNet is the first SSO-based algorithm to improve LeNet. SSO-LeNet outperforms LeNet in solution quality with a significant improvement in test time. SSO-LeNet outperforms PSO-LeNet in solution quality. SSO-LeNet has considerable potential for improving AlexNet, GoogLeNet, etc. efficiently. Abstract: Convolutional neural networks (CNNs) are widely used in image recognition. Numerous CNN models, such as LeNet, AlexNet, VGG, ResNet, and GoogLeNet, have been developed by increasing the number of layers to improve performance. However, performance deteriorates beyond a certain number of layers. Hence, hyperparameter optimization is a more efficient way to improve CNNs. To validate this concept, in the present study, an algorithm based on simplified swarm optimization was developed for optimizing the hyperparameters of the simplest CNN model: LeNet. The results of experiments involving the MNIST, Fashion-MNIST, and CIFAR-10 datasets indicated that the accuracy of the proposed algorithm was higher than those of LeNet and PSO-LeNet and that the proposed algorithm can be applied to more complex models such as AlexNet.