EKConv: Compressing Convolutional Neural Networks with Evolutionary Kernel Convolution. Issue 1 (1st February 2023)

• Record Type: Journal Article
• Authors: Yang, Dengjie; Chen, Zhiwei; Sun, Yi; He, Qing; Ye, Shiwei; Chen, Deyuan
• Is Part Of: Journal of physics. Volume 2425:Issue 1(2023)
• Publication Date: 2023-02-01
• Subjects: Convolutional neural networks -- EKConv -- Compress the parameters -- Image classification; Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/2425/1/012011
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: Convolutional neural networks (CNNs) have achieved tremendous success in visual recognition tasks but mainly rely on massive learnable parameters. To solve this problem, many effective and efficient convolution operators have been proposed, such as group-wise convolution, point-wise convolution, and depth-wise convolution. However, the above convolution operations model and optimize the weight relationship within the same convolutional layer. To reduce the network parameters, we explicitly construct the relationship between convolution kernels of adjacent layers. Specifically, we propose an evolutionary kernel convolution, namely EKConv, to generate weight parameters by group-wise convolution efficiently. In particular, EKConv makes the kernel parameters of the current convolutional layer inherit from its preceding adjacent kernel, which promotes the information exchange between convolution kernels. More importantly, EKConv is a novel plug-and-play module that can be easily embedded into mainstream CNNs. Extensive experimental results show that EKConv can compress the parameters of CNNs by a large margin yet barely sacrifice image classification performance.