DAFA-BiLSTM: Deep Autoregression Feature Augmented Bidirectional LSTM network for time series prediction. (January 2023)

• Record Type: Journal Article
• Authors: Wang, Heshan; Zhang, Yiping; Liang, Jing; Liu, Lili
• Is Part Of: Neural networks. Volume 157(2023)
• Page Start: 240
• Page End: 256
• Publication Date: 2023-01
• Subjects: Time series prediction -- Long short-term memory -- Deep recurrent neural network -- Feature augmented -- Vector autoregression transformation; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2022.10.009
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: Time series forecasting models that use the past information of exogenous or endogenous sequences to forecast future series play an important role in the real world because most real-world time series datasets are rich in time-dependent information. Most conventional prediction models for time series datasets are time-consuming and fraught with complex limitations because they usually fail to adequately exploit the latent spatial dependence between pairs of variables. As a successful variant of recurrent neural networks, the long short-term memory network (LSTM) has been demonstrated to have stronger nonlinear dynamics to store sequential data than traditional machine learning models. Nevertheless, the common shallow LSTM architecture has limited capacity to fully extract the transient characteristics of long interval sequential datasets. In this study, a novel deep autoregression feature augmented bidirectional LSTM network (DAFA-BiLSTM) is proposed as a new deep BiLSTM architecture for time series prediction. Initially, the input vectors are fed into a vector autoregression (VA) transformation module to represent the time-delayed linear and nonlinear properties of the input signals in an unsupervised way. Then, the learned nonlinear combination vectors of VA are progressively fed into different layers of BiLSTM and the output of the previous BiLSTM module is also concatenated with the time-delayed linear vectors of the VA as an augmented feature to form new additional input signals for the next adjacent BiLSTM layer. Extensive real-world time series applications are addressed to demonstrate the superiority and robustness of the proposed DAFA-BiLSTM. Comparative experimental results and statistical analysis show that the proposed DAFA-BiLSTM has good adaptive performance as well as robustness even in noisy environment. Highlights: We combine the advantages of a pretraining VA mechanism and deep BiLSTM in the proposed DAFA-BiLSTM model to efficiently represent the linear and nonlinear features, and learn the nonlinear feature information from different directions while generating a hierarchical feature representation. The DAFA-BiLSTM model can effectively learn the multi-featured scale information of time series datasets and allows the model to simultaneously train the time series in both directions. Therefore, the proposed DAFA-BiLSTM is more robust to deal with time series forecasting. The performance and generalization of DAFA-BiLSTM is evaluated by extensive real-world time series benchmarks. The validation tests and statistical analysis can explain why the DAFA-BiLSTM model achieves satisfactory performance and astonishing results, accordingly, the interpretability of the proposed model is also demonstrated.