Discriminative and regularized echo state network for time series classification. (October 2022)

• Record Type: Journal Article
• Authors: Wang, Heshan; Liu, Yuxi; Wang, Dongshu; Luo, Yong; Tong, Chudong; Lv, Zhaomin
• Is Part Of: Pattern recognition. Volume 130(2022)
• Publication Date: 2022-10
• Subjects: Echo state network -- Recurrent neural networks -- Discriminative feature extraction -- Time series classification -- Outlier-robust weights; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108811
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a DFA algorithm which uses the linear combinations of input samples features to regenerate the input weights of Echo State Network (ESN) instead of using the original random input weights. The input weights of our final ESN classifiers take the relative importance of temporal data at all classes into consideration. To enhance the stability of the output weights, an ORW algorithm is proposed based on the training errors for classification tasks. The training errors are then used to iterate backwards to train the loss function and regularization term for obtaining a more compact output layer architecture. The performance of our DR-ESN classifier on massive benchmarks indicate that the proposed DR-ESN can considerably improve the original ESN classifier and that our DR-ESN classifier yields comparable performance compared with some state-of-the-art classifiers. The input weights and output weights visualization results demonstrate the superiority of DR-ESN. Abstract: An echo State Network (ESN) is a special structure of a recurrent neural network (RNN) in which the recurrent neurons are randomly connected. ESN models which have achieved a high accuracy on time series prediction tasks can be used as time series prediction models in many domains. Nevertheless, in most ESN models, the input weights are randomly generated and the output weights calculated by the least square method are susceptible to outliers, which cannot guarantee that the ESN models will always be optimal for a given task. In this paper, a novel discriminative and regularized ESN (DR-ESN) combines discriminative feature aggregation (DFA) and outlier-robust weights (ORW) algorithms are proposed for time series classification. DFA is firstly proposed to replace the random input weights of ESN with the constrained weights generated from sample information. In DFA, weight vectors are selected from the vector space spanned by initial input sequence vectors, then the new generated input weights can adequately represent the data features. Secondly, ORW is employed to enhance the robustness of output weights by constraining the weights assigned to samples with large training errors. The weights evaluation and experiments on a massive set of the synthetic time series data, real-world bearing fault data and UCR benchmarks indicate that the proposed DR-ESN can not only considerably improve the original ESN classifier but also effectively suppress the effect of outliers on classification performance.