Multi-step forecasting of multivariate time series using multi-attention collaborative network. (January 2023)

• Record Type: Journal Article
• Authors: He, Xiaoyu; Shi, Suixiang; Geng, Xiulin; Yu, Jie; Xu, Lingyu
• Is Part Of: Expert systems with applications. Volume 211(2023)
• Publication Date: 2023-01
• Subjects: Multivariate time series -- Multi-attention -- Deep neural network -- Multi-step forecasting; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.118516
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Multi-step forecasting of multivariate time series plays a critical role in many fields, such as disaster warning and financial analysis. While attention-based recurrent neural networks (RNNs) achieved encouraging performance, two limitations exist in current models: i) Existing approaches merely focus on variables' interactions, and ignore the negative noise of non-predictive variables, ii) These methods cannot model the difference in the temporal importance of the target and the non-predictive series to prediction. To tackle these challenges, we propose a triangle structured Multi-Attention Collaborative Network (MACN), which includes a backbone network T-net with attention-based encoder–decoder framework, and an auxiliary hierarchical network NP-net. NP-net focuses on non-predictive variables, capturing the most relevant variables and temporal dependencies through the proposed variables-distillation attention network (VDN) and long short-term memory network (LSTM). T-net executes on target variable, and its encoder and decoder are both connected to NP-net, thereby using the output of NP-net to assist learning and decision-making. Specifically we design a knowledge-enhanced LSTM (KeLSTM) as the encoder and decoder of T-net. In the coding stage, KeLSTM refines the output of NP-net to strengthen the latent semantics of the target variable. In the decoding stage, KeLSTM captures subtle differences between the target and the non-predictive variables' contribution to prediction, and improves model's predictive ability by alleviating such conflicts. Experiments on three real-world datasets demonstrate that MACN outperforms different types of state-of-the-art methods. Highlights: A triangle structured multi-attention model for multivariate time series prediction. Variables-distillation attention selects the most relevant non-predictive variables. The KeLSTM uses non-predictive variables to enrich the target variable's semantics. The model distinguishes the effects of target and non-predictive variables on tasks. MACN is competitive in multivariate time series multi-step prediction.