Reduced‐gate convolutional long short‐term memory using predictive coding for spatiotemporal prediction. (22nd January 2020)

• Record Type: Journal Article
• Authors: Elsayed, Nelly; Maida, Anthony S.; Bayoumi, Magdy
• Is Part Of: Computational intelligence. Volume 36:Number 3(2020)
• Page Start: 910
• Page End: 939
• Publication Date: 2020-01-22
• Subjects: convolutional long short‐term memory -- convLSTM -- predictive coding -- rgcLSTM -- spatiotemporal prediction -- video prediction; Artificial intelligence -- Periodicals; Computational linguistics -- Periodicals; 006.3
• DOI: 10.1111/coin.12277
• Languages: English
• ISSNs: 0824-7935

Abstract

Abstract: Spatiotemporal sequence prediction is an important problem in deep learning. We study next‐frame(s) video prediction using a deep‐learning‐based predictive coding framework that uses convolutional LSTM (convLSTM) modules. We introduce a novel rgcLSTM architecture that requires a significantly lower parameter budget than a comparable convLSTM. By using a single multifunction gate, our reduced‐gate model achieves equal or better next‐frame(s) prediction accuracy than the original convolutional LSTM while using a smaller parameter budget, thereby reducing training time and memory requirements. We tested our reduced gate modules within a predictive coding architecture on the moving MNIST and KITTI datasets. We found that our reduced‐gate model has a significant reduction of approximately 40% of the total number of training parameters and a 25% reduction in elapsed training time in comparison with the standard convolutional LSTM model. The performance accuracy of the new model was also improved. This makes our model more attractive for hardware implementation, especially on small devices. We also explored a space of 20 different gated architectures to get insight into how our rgcLSTM fits into that space.