Adaptive stochastic resonance based convolutional neural network for image classification. (September 2022)

• Record Type: Journal Article
• Authors: Duan, Lingling; Ren, Yuhao; Duan, Fabing
• Is Part Of: Chaos, solitons and fractals. Volume 162(2022)
• Publication Date: 2022-09
• Subjects: Adaptive stochastic resonance -- Convolutional neural network -- Threshold activation function -- Noise injection -- Image classification; Chaotic behavior in systems -- Periodicals; Solitons -- Periodicals; Fractals -- Periodicals; Chaotic behavior in systems; Fractals; Solitons; Periodicals; 003.7
• DOI: 10.1016/j.chaos.2022.112429
• Languages: English
• ISSNs: 0960-0779

Abstract

Abstract: In this paper, we exploit the adaptive stochastic resonance effect in the convolutional neural network with threshold activation functions for enabling the back-propagation gradient computation. During training, the injection of noise into the threshold activation function makes it to be differentiable at forward pass, thus exact gradients of the loss function with respect to network parameters including the injected noise level can smoothly propagate at backforward pass. This training method effectively uses the non-zero noise to improve the performance of the designed convolutional neural network. In the testing phase, the differentiable activation function is decoupled into a set of threshold functions driven by mutually independent noise components, which endows a hardware-friendly feature of the designed neural network. Experiments show that the designed low-precision convolutional neural network has a comparable performance with the full-precision one with ReLU activation functions on two benchmark data sets. These results also extend the potential application of adaptive stochastic resonance into the convolutional neural network for image classification. Highlights: Exploring adaptive stochastic resonance effect in threshold convolutional neural networks Propagating exact gradients at backforward pass via the noise-smoothed neurons Enhancing the testing accuracy of the convolutional neural networks by the noise injection