Twin delayed deep deterministic policy gradient-based deep reinforcement learning for energy management of fuel cell vehicle integrating durability information of powertrain. (15th December 2022)

• Record Type: Journal Article
• Authors: Zhang, Yuanzhi; Zhang, Caizhi; Fan, Ruijia; Huang, Shulong; Yang, Yun; Xu, Qianwen
• Is Part Of: Energy conversion and management. Volume 274(2022)
• Publication Date: 2022-12-15
• Subjects: Deep reinforcement learning -- Energy management strategy -- Fuel cell vehicle -- Fuel economy -- Lifespan durability -- Twin delayed deep deterministic policy gradient; Direct energy conversion -- Periodicals; Energy storage -- Periodicals; Energy transfer -- Periodicals; Énergie -- Conversion directe -- Périodiques; Direct energy conversion; Periodicals; 621.3105
• DOI: 10.1016/j.enconman.2022.116454
• Languages: English
• ISSNs: 0196-8904

Abstract

Highlights: Learning-based strategy deals with continuous action control of energy management. The transient degradation of fuel cell and battery is incorporated into reward. Generalization of on-line control is verified under various driving scenarios. Fuel consumption and degradation reduction of fuel cell and battery. Abstract: Deep reinforcement learning (DRL)-based energy management strategy (EMS) is attractive for fuel cell vehicle (FCV). Nevertheless, the fuel economy and lifespan durability of proton exchange membrane fuel cell (PEMFC) stack and lithium-ion battery (LIB) may not be synchronously optimized since transient degradation variations of PEMFC stack and LIB are not generally regarded for DRL-based EMSs. Furthermore, the inappropriate action space and the overestimated value function of DRL can lead to suboptimal EMS for on-line control. To this end, the objective of this research endeavors to formulate a twin delayed deep deterministic policy gradient (TD3)-based EMS integrating durability information of PEMFC stack and LIB, which can interact with the vehicle operating states to continuously control the hybrid powertrain and limit the overestimation of DRL value function for ensuring maximum multi-objective reward at each moment. Unlike traditional DRL-based EMSs, the multi-objective reward function for this study is enlarged to incorporate the hydrogen consumption, state of charge (SOC)-sustaining penalty and transient lifespan degradation information of PEMFC stack and LIB in off-line training and on-line control. The results demonstrate that the proposed EMS can drastically lessen the training time and computational burden. Meanwhile, in contrast with deep Q-network (DQN)-based and deep deterministic policy gradient (DDPG)-based EMSs in the various real-world urban and standard driving cycles, the proposed EMS can achieve hydrogen abatement at least 9.76% and 1.07%, and slow down total powertrain degradation at least 9.11% and 2.62%, respectively.