Multi-step reinforcement learning for model-free predictive energy management of an electrified off-highway vehicle. (1st December 2019)

• Record Type: Journal Article
• Authors: Zhou, Quan; Li, Ji; Shuai, Bin; Williams, Huw; He, Yinglong; Li, Ziyang; Xu, Hongming; Yan, Fuwu
• Is Part Of: Applied energy. Volume 255(2019)
• Publication Date: 2019-12-01
• Subjects: Model-free predictive control -- Energy management -- Multi-step reinforcement learning -- Markov decision problem -- Hybrid electric vehicle; Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2019.113755
• Languages: English
• ISSNs: 0306-2619

Abstract

Highlights: Reinforcement Learning is researched for energy saving in a hybrid vehicle. Energy efficiency can be continuously improved by multi-step learning. The 'Recurrent-to-Terminal' strategy is shown the most effective learning strategy. The method can save energy by more than 7.8% in the selected real-time operations. Abstract: The energy management system of an electrified vehicle is one of the most important supervisory control systems which manages the use of on-board energy resources. This paper researches a 'model-free' predictive energy management system for a connected electrified off-highway vehicle. A new reinforcement learning algorithm with the capability of 'multi-step' learning is proposed to enable the all-life-long online optimisation of the energy management control policy. Three multi-step learning strategies (Sum-to-Terminal, Average-to-Neighbour Recurrent-to-Terminal) are researched for the first time. Hardware-in-the-loop tests are carried out to examine the control functionality for real application of the proposed 'model-free' method. The results show that the proposed method can continuously improve the vehicle's energy efficiency during the real-time hardware-in-the-loop test, which increased from the initial level of 34% to 44% after 5 h' 35-step learning. Compared with a well-designed model-based predictive energy management control policy, the model-free predictive energy management method can increase the prediction horizon length by 71% (from 35 to 65 steps with 1 s interval in real-time computation) and can save energy by at least 7.8% for the same driving conditions.