Reliability evaluation of FPGA based pruned neural networks. (March 2022)

• Record Type: Journal Article
• Authors: Gao, Zhen; Yao, Yi; Wei, Xiaohui; Yan, Tong; Zeng, Shulin; Ge, Guangjun; Wang, Yu; Ullah, Anees; Reviriego, Pedro
• Is Part Of: Microelectronics and reliability. Volume 130(2022)
• Publication Date: 2022-03
• Subjects: Convolutional Neural Networks (CNNs) -- Pruning -- Reliability -- FPGAs -- Fault injection; Electronic apparatus and appliances -- Reliability -- Periodicals; Miniature electronic equipment -- Periodicals; Appareils électroniques -- Fiabilité -- Périodiques; Équipement électronique miniaturisé -- Périodiques; Electronic apparatus and appliances -- Reliability; Miniature electronic equipment; Periodicals; 621.3815
• DOI: 10.1016/j.microrel.2022.114498
• Languages: English
• ISSNs: 0026-2714

Abstract

Abstract: Convolutional Neural Networks (CNNs) are widely used for image classification. To fit the implementation of CNNs on resource-limited systems like FPGAs, pruning is a popular technique to reduce the complexity. In this paper, the robustness of the pruned CNNs against errors on weights and configuration memory of the FPGA accelerator is evaluated with VGG16 as a case study, and two popular pruning methods (magnitude-based and filter pruning) are considered. In particular, the accuracy loss of the original VGG16 and the ones with different pruning rates is tested based on fault injection experiments, and the results show that the effect of errors on weights and configuration memories are different for the two pruning methods. For errors on weights, the networks pruned using both methods demonstrate higher reliability with higher pruning rates, but the ones using filter pruning are relatively less reliable. For errors on configuration memory, errors on about 30% of the configuration bits will affect the CNN operation, and only 14% of them will introduce significant accuracy loss. However, the effect of the same critical bits is different for the two pruning methods. The pruned networks using magnitude-based method are less reliable than the original VGG16, but the ones using filter pruning are more reliable than the original VGG16. The different effects are explained based on the structure of the CNN accelerator and the properties of the two pruning methods. The impact of quantization on the CNN reliability is also evaluated for the magnitude-based pruning method. Highlights: Hardware fault injection for reliability evaluation of FPGA based pruned CNNs Reliability of CNNs with different pruning rates to faults on weights for two pruning methods Reliability of CNNs with different pruning rates to faults on weights of different layers Reliability of CNNs with different pruning rates to faults on configuration memory Separate reliability evaluation for faults on the DSP array and adder tree in the accelerator