Sparse CapsNet with explicit regularizer. (April 2022)

• Record Type: Journal Article
• Authors: Shi, Ruiyang; Niu, Lingfeng; Zhou, Ruizhi
• Is Part Of: Pattern recognition. Volume 124(2022)
• Publication Date: 2022-04
• Subjects: Capsule network -- Model compression -- Sparse regularization -- Proximal gradient descent; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108486
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel sparse capsule network (CapsNet) with explicit regularizer is proposed. Both component-wise and filter-wise sparsity are considered for weight compression. A new stochastic proximal algorithm is designed to train the sparse CapsNet. Numerical experiments validate the effectiveness and efficiency of our method. Abstract: Capsule Network (CapsNet) achieves great improvements in recognizing pose and deformation through a novel encoding mode. However, it carries a large number of parameters, leading to the challenge of heavy memory and computational cost. To solve this problem, we propose sparse CapsNet with an explicit regularizer in this paper. To our knowledge, it's the first work that utilizes sparse optimization to compress CapsNet. Specifically, to reduce unnecessary weight parameters, we first introduce the component-wise absolute value regularizer into the objective function of CapsNet based on zero-means Laplacian prior. Then, to reduce the computational cost and speed up CapsNet, the weight parameters are further grouped by 2D filters and sparsified by 1-norm regularization. To train our model efficiently, a new stochastic proximal gradient algorithm, which has analytical solutions at each iteration, is presented. Extensive numerical experiments on four commonly used datasets validate the effectiveness and efficiency of the proposed method.