Cost-effective stochastic MAC circuits for deep neural networks. (September 2019)

• Record Type: Journal Article
• Authors: Sim, Hyeonuk; Lee, Jongeun
• Is Part Of: Neural networks. Volume 117(2019)
• Page Start: 152
• Page End: 162
• Publication Date: 2019-09
• Subjects: Stochastic computing -- Convolutional neural network -- Stochastic number generator -- Hardware acceleration -- Low-discrepancy code -- Variable latency; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2019.04.017
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Stochastic computing (SC) is a promising computing paradigm that can help address both the uncertainties of future process technology and the challenges of efficient hardware realization for deep neural networks (DNNs). However the impreciseness and long latency of SC have rendered previous SC-based DNN architectures less competitive against optimized fixed-point digital implementations, unless inference accuracy is significantly sacrificed. In this paper we propose a new SC-MAC (multiply-and-accumulate) algorithm, which is a key building block for SC-based DNNs, that is orders of magnitude more efficient and accurate than previous SC-MACs. We also show how our new SC-MAC can be extended to a vector version and used to accelerate both convolution and fully-connected layers of convolutional neural networks (CNNs) using the same hardware . Our experimental results using CNNs designed for MNIST and CIFAR-10 datasets demonstrate that not only is our SC-based CNNs more accurate and 40 ∼ 490 × more energy-efficient for convolution layers than conventional SC-based ones, but ours can also achieve lower area–delay product and lower energy compared with precision-optimized fixed-point implementations without sacrificing accuracy. We also demonstrate the feasibility of our SC-based CNNs through FPGA prototypes.