Bidirectional stochastic configuration network for regression problems. (August 2021)

• Record Type: Journal Article
• Authors: Cao, Weipeng; Xie, Zhongwu; Li, Jianqiang; Xu, Zhiwu; Ming, Zhong; Wang, Xizhao
• Is Part Of: Neural networks. Volume 140(2021)
• Page Start: 237
• Page End: 246
• Publication Date: 2021-08
• Subjects: Stochastic configuration network -- Random vector functional link network -- Randomized algorithms -- Neural networks with random weights -- Constructive neural networks; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.03.016
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: To adapt to the reality of limited computing resources of various terminal devices in industrial applications, a randomized neural network called stochastic configuration network (SCN), which can conduct effective training without GPU, was proposed. SCN uses a supervisory random mechanism to assign its input weights and hidden biases, which makes it more stable than other randomized algorithms but also leads to time-consuming model training. To alleviate this problem, we propose a novel bidirectional SCN algorithm (BSCN) in this paper, which divides the way of adding hidden nodes into two modes: forward learning and backward learning. In the forward learning mode, BSCN still uses the supervisory mechanism to configure the parameters of the newly added nodes, which is the same as SCN. In the backward learning mode, BSCN calculates the parameters at one time based on the residual error feedback of the current model. The two learning modes are performed iteratively until the prediction error of the model reaches an acceptable level or the number of hidden nodes reaches its maximum value. This semi-random learning mechanism greatly speeds up the training efficiency of the BSCN model and significantly improves the quality of the hidden nodes. Extensive experiments on ten benchmark regression problems, two real-life air pollution prediction problems, and a classical image processing problem show that BSCN can achieve faster training speed, higher stability, and better generalization ability than SCN. Highlights: A novel bidirectional stochastic configuration network (BSCN) was proposed to solve regression problems in this paper. BSCN can greatly accelerate the training efficiency of the SCN model and make the model better in generalization ability and more compact in network structure. The effectiveness of BSCN has been verified on extensive experiments. BSCN provides a stable and fast modeling solution for platforms with limited computing ability.