Mixed-precision quantized neural networks with progressively decreasing bitwidth. (March 2021)

• Record Type: Journal Article
• Authors: Chu, Tianshu; Luo, Qin; Yang, Jie; Huang, Xiaolin
• Is Part Of: Pattern recognition. Volume 111(2021)
• Publication Date: 2021-03
• Subjects: Model compression -- Quantized neural networks -- Mixed-precision; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107647
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We address the trade-off issue between aggressive model compression and the superior performance of quantized neural networks. Based on the observation on internal feature distributions, a mixed-precision QNN with progressively decreasing bitwidth is proposed. A heuristic of bitwidth assignment based on the quantitative separability for feature representation is given. Several typical CNNs including AlexNex, ResNet and Faster R-CNN are quantized based on the proposed mixed-precision method. The experimental results demonstrate that the mixed-precision networks could achieve preferable performance with less memory space. Abstract: Efficient model inference is an important and practical issue in the deployment of deep neural networks on resource constraint platforms. Network quantization addresses this problem effectively by leveraging low-bit representation and arithmetic that could be conducted on dedicated embedded systems. In the previous works, the parameter bitwidth is set homogeneously and there is a trade-off between superior performance and aggressive compression. Actually, the stacked network layers, which are generally regarded as hierarchical feature extractors, contribute diversely to the overall performance. For a well-trained neural network, the feature distributions of different categories are organized gradually as the network propagates forward. Hence the capability requirement on the subsequent feature extractors is reduced. It indicates that the neurons in posterior layers could be assigned with lower bitwidth for quantized neural networks. Based on this observation, a simple yet effective mixed-precision quantized neural network with progressively decreasing bitwidth is proposed to improve the trade-off between accuracy and compression. Extensive experiments on typical network architectures and benchmark datasets demonstrate that the proposed method could achieve better or comparable results while reducing the memory space for quantized parameters by more than 25% in comparison with the homogeneous counterparts. In addition, the results also demonstrate that the higher-precision bottom layers could boost the 1-bit network performance appreciably due to a better preservation of the original image information while the lower-precision posterior layers contribute to the regularization of k − bit networks.