Echo state network activation function based on bistable stochastic resonance. (December 2021)

• Record Type: Journal Article
• Authors: Liao, Zhiqiang; Wang, Zeyu; Yamahara, Hiroyasu; Tabata, Hitoshi
• Is Part Of: Chaos, solitons and fractals. Volume 153:Part 2(2021)
• Publication Date: 2021-12
• Subjects: Stochastic resonance -- Noisy adaptability -- Echo state network -- Short-term memory -- Activation function -- Reservoir computing; Chaotic behavior in systems -- Periodicals; Solitons -- Periodicals; Fractals -- Periodicals; Chaotic behavior in systems; Fractals; Solitons; Periodicals; 003.7
• DOI: 10.1016/j.chaos.2021.111503
• Languages: English
• ISSNs: 0960-0779

Abstract

Abstract: Stochastic resonance (SR) is a phenomenon wherein an information-carrying signal is enhanced via noise in a nonlinear system. This phenomenon enables living beings to adapt to noisy environments and use environmental noise to obtain useful information. A novel activation function of the echo state network (ESN) based on bistable SR is proposed in this study. Instead of using the tanh activation function—which is representative of the traditional threshold activation function—the bistable SR activation function is used to improve the noise adaptability of the ESN. Further, the proposed activation function provides a short-term memory (STM) ability that is not provided by the widely used threshold activation function, and thus, a physical reservoir can be designed using the proposed function. An STM task and a parity check task are used to verify the short-term memory and nonlinear ability of the bistable SR activation function. Further, two different prediction benchmarks prove that the proposed activation function can improve the noise adaptability of ESN. Finally, a visual recognition task is performed to demonstrate the potential of the SR activation function for physical reservoir computing.