A comparative study of 13 deep reinforcement learning based energy management methods for a hybrid electric vehicle. (1st March 2023)

• Record Type: Journal Article
• Authors: Wang, Hanchen; Ye, Yiming; Zhang, Jiangfeng; Xu, Bin
• Is Part Of: Energy. Volume 266(2023)
• Publication Date: 2023-03-01
• Subjects: Hybrid electric vehicle -- Energy management strategy -- Deep reinforcement learning; Power resources -- Periodicals; Power (Mechanics) -- Periodicals; Energy consumption -- Periodicals; 333.7905
• DOI: 10.1016/j.energy.2022.126497
• Languages: English
• ISSNs: 0360-5442

Abstract

Abstract: Energy management strategy (EMS) has a huge impact on the energy efficiency of hybrid electric vehicles (HEVs). Recently, fast-growing number of studies have applied different deep reinforcement learning (DRL) based EMS for HEVs. However, a unified performance review benchmark is lacking for most popular DRL algorithms. In this study, 13 popular DRL algorithms are applied as HEV EMSs. The reward performance, computation cost, and learning convergence of different DRL algorithms are discussed. In addition, HEV environments are modified to fit both discrete and continuous action spaces. The results show that the stability of agent during the learning process of continuous action space is more stable than discrete action space. In the continuous action space, SAC has the highest reward, and PPO has the lowest time cost. In discrete action space, DQN has the lowest time cost, and FQF has the highest reward. The comparison among SAC, FQF, rule-based, and equivalent consumption minimization strategies (ECMS) shows that DRL EMSs run the engine more efficiently, thus saving fuel consumption. The fuel consumption of FQF is 10.26% and 5.34% less than Rule-based and ECMS, respectively. The contribution of this paper will speed up the application of DRL algorithms in the HEV EMS application. Highlights: A propulsion model is designed for a Hybrid Electric Vehicle in python and the entire code is shared in Github. 13 popular DRL algorithms are introduced and both discrete and continuous action space algorithms are considered. Key measures, including convergence reward/episode/time and final test result, are compared. The best DRL algorithms of each action space type are compared with baseline EMS.