DQL energy management: An online-updated algorithm and its application in fix-line hybrid electric vehicle. (15th June 2021)

• Record Type: Journal Article
• Authors: Zou, Runnan; Fan, Likang; Dong, Yanrui; Zheng, Siyu; Hu, Chenxing
• Is Part Of: Energy. Volume 225(2021)
• Publication Date: 2021-06-15
• Subjects: Online-updated energy management strategy -- Prioritized replay -- Model predictive control -- Deep Q learning -- Hybrid electric vehicle; Power resources -- Periodicals; Power (Mechanics) -- Periodicals; Energy consumption -- Periodicals; 333.7905
• DOI: 10.1016/j.energy.2021.120174
• Languages: English
• ISSNs: 0360-5442

Abstract

Abstract: With decades' development of energy management strategy in hybrid electric vehicle, learning-based method has been deemed as a key solution for energy economy and real time. However, current energy management strategy cannot reach an optimal energy economy performance and online update in a tolerable time lag. Aiming at solving these problems, an accelerated reinforcement learning method and an online-updated strategy are proposed in present work. Firstly, prioritized replay is applied in deep Q network with normalized advantage function for a fast convergence to an optimal strategy. Prioritized replay module endows weight to trained history data which is utilized in neural network training. The neural network is updated towards optimal strategy by weight in an effective way. Secondly, the online-updated strategy for fix-line hybrid electric vehicle is designed based on the accelerated reinforcement learning method and model predictive control. The predicted future road information generated by model predictive control in each time interval is delivered to the accelerated reinforcement learning module for online energy management strategy generating. Finally, with all efforts above, the online-updated strategy is carried out and validated through hardware-in-the-loop simulation. The results show that this approach promotes the energy economic performance while updating strategy in real time. Highlights: Mathematical model for a fix-line HEV considering engine, battery and vehicle dynamic. An accelerated deep Q learning energy management strategy with prioritized replay. An online energy management strategy based on the accelerated deep Q learning. Hardware-in-the-loop simulation for validation of the online energy management. Effectiveness of the proposed strategies in improving fuel and time economy.