Efficient network architecture search via multiobjective particle swarm optimization based on decomposition. (March 2020)

• Record Type: Journal Article
• Authors: Jiang, Jing; Han, Fei; Ling, Qinghua; Wang, Jie; Li, Tiange; Han, Henry
• Is Part Of: Neural networks. Volume 123(2020)
• Page Start: 305
• Page End: 316
• Publication Date: 2020-03
• Subjects: Convolutional neural network -- Neural architecture search -- Multiobjective particle swarm optimization -- Decomposition; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2019.12.005
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: The efforts devoted to manually increasing the width and depth of convolutional neural network (CNN) usually require a large amount of time and expertise. It has stimulated a rising demand of neural architecture search (NAS) over these years. However, most popular NAS approaches solely optimize for low prediction error without penalizing high structure complexity. To this end, this paper proposes MOPSO/D-Net, a CNN architecture search method with multiobjective particle swarm optimization based on decomposition (MOPSO/D). The main goal is to reformulate NAS as a multiobjective evolutionary optimization problem, where the optimal architecture is learned by minimizing two conflicting objectives, namely the error rate of classification and number of parameters of the network. Along with the hybrid binary encoding and adaptive penalty-based boundary intersection, an improved MOPSO/D is further proposed to solve the formulated multiobjective NAS and provide diverse tradeoff solutions. Experimental studies verify the effectiveness of MOPSO/D-Net compared with current manual and automated CNN generation methods. The proposed algorithm achieves impressive classification performance with a small number of parameters on each of two benchmark datasets, particularly, 0.4% error rate with 0.16 M params on MNIST and 5.88% error rate with 8.1 M params on CIFAR-10, respectively.