What is the best RNN-cell structure to forecast each time series behavior?. (1st April 2023)

• Record Type: Journal Article
• Authors: Khaldi, Rohaifa; El Afia, Abdellatif; Chiheb, Raddouane; Tabik, Siham
• Is Part Of: Expert systems with applications. Volume 215(2023)
• Publication Date: 2023-04-01
• Subjects: Forecasting -- Time series -- Times series behavior -- RNN models -- LSTM cells -- Performance evaluation metrics; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.119140
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: It is unquestionable that time series forecasting is of paramount importance in many fields. The most used machine learning models to address time series forecasting tasks are Recurrent Neural Networks (RNNs). Typically, those models are built using one of the three most popular cells, ELMAN, Long–Short Term Memory (LSTM), or Gated Recurrent Unit (GRU) cells, each cell has a different structure and implies a different computational cost. However, it is not clear why and when to use each RNN-cell structure. Actually, there is no comprehensive characterization of all the possible time series behaviors and no guidance on what RNN cell structure is the most suitable for each behavior. The objective of this study is two-fold: it presents a comprehensive taxonomy of all possible time series behaviors (deterministic, random-walk, nonlinear, long-memory, and chaotic), and provides insights into the best RNN cell structure for each time series behavior. We conducted two experiments: (1) The first experiment evaluates and analyzes the role of each component in the LSTM-Vanilla cell by creating 11 variants based on one alteration in its basic architecture (removing, adding, or substituting one cell component). (2) The second experiment evaluates and analyzes the performance of 20 possible RNN-cell structures. To evaluate, compare, and select the best model, different statistical metrics were used: error-based metrics, information criterion-based metrics, naïve-based metric, and direction change-based metric. To further improve our confidence in the models' interpretation and selection, Friedman Wilcoxon–Holm signed-rank test was used. Our results advocate the usage and the exploration of the newly created RNN variant, named SLIM, in time series forecasting thanks to its high ability to accurately predict the different time series behaviors as well as its simple structural design that does not require expensive temporal and computing resources. Highlights: A taxonomy of time series behaviors was proposed. A taxonomy of Recurrent Neural Network (RNN) cell structures was proposed. A set of 31 RNN cell structures was evaluated. The utility of each component in the Long–Short Term Memory cell was evaluated. The best cell structure for forecasting each time series behavior was provided.