Using checksum to improve the reliability of embedded convolutional neural networks. (September 2022)

• Record Type: Journal Article
• Authors: Liu, Zhi; Deng, Zhen; Yang, Xinni
• Is Part Of: Microelectronics and reliability. Volume 136(2022)
• Publication Date: 2022-09
• Subjects: CNN -- DNN -- Soft error -- Error resilience; Electronic apparatus and appliances -- Reliability -- Periodicals; Miniature electronic equipment -- Periodicals; Appareils électroniques -- Fiabilité -- Périodiques; Équipement électronique miniaturisé -- Périodiques; Electronic apparatus and appliances -- Reliability; Miniature electronic equipment; Periodicals; 621.3815
• DOI: 10.1016/j.microrel.2022.114666
• Languages: English
• ISSNs: 0026-2714

Abstract

Abstract: Convolutional Neural Networks (CNNs) have been used in resource-constrained edge devices, including safety-critical applications. However, recent studies demonstrate that soft errors may result in CNN prediction failures. Many approaches are proposed at the end of layers to detect computation errors. Still, these approaches incur severe overheads or cannot detect errors in memory, e.g., traditional Double Modular Redundancy (DMR) can protect data integrity in memory, but it incurs significant overheads on time and memory. Some existing DNN-specific approaches are lightweight to protect DNNs from transient faults but cannot provide high fault coverage to detect permanent faults in memory. Thus, we propose two low-cost software approaches, channel-based weight checksum and layer-based output checksum, which protect the integrity of weights and layer outputs in memory. We perform a large number of fault injections to evaluate the performance of the proposed approaches and compare them with other approaches. The results show that combining the proposed approaches can provide approximately 100 % fault coverage for weights and layer outputs in memory. The combination increases total time and memory overheads by 4 % and 17 %, respectively, for DNNs that run sequentially. In addition, we observe that the layer-based output checksum can improve the error resilience of register files because most SDCs caused by register file errors are induced by the control flow error and the pointer error. Finally, we present a case study on MobileNet to show that our approaches can be generalized to other DNN libraries that run parallel by dividing the single-layer execution into N parts. When N is 2, our approaches increase total time and memory overheads by 7.4 % and 31 %, respectively. Highlights: Rank the error resilience of layers' weights and intermediate data in memory for DNNs. Propose two lightweight checksum approaches to protect the weights and intermediate data in memory for embedded CNNs. Observe that a considerable part of critical SDCs caused by register errors are induced by control-flow errors. Present an insightful comparison on protecting embedded CNNs' memory among the existing DNN-specific, DMR, and our proposed approaches.