Deep reinforcement learning-based energy management of hybrid battery systems in electric vehicles. (April 2021)

• Record Type: Journal Article
• Authors: Li, Weihan; Cui, Han; Nemeth, Thomas; Jansen, Jonathan; Ünlübayir, Cem; Wei, Zhongbao; Zhang, Lei; Wang, Zhenpo; Ruan, Jiageng; Dai, Haifeng; Wei, Xuezhe; Sauer, Dirk Uwe
• Is Part Of: Journal of energy storage. Volume 36(2021)
• Publication Date: 2021-04
• Subjects: Lithium-ion battery -- Hybrid battery system -- Reinforcement learning -- Deep Q-learning -- Energy management -- Electric vehicle; Energy storage -- Periodicals; Energy storage -- Research -- Periodicals; 621.3126
• DOI: 10.1016/j.est.2021.102355
• Languages: English
• ISSNs: 2352-152X

Abstract

Abstract: In this paper, we propose an energy management strategy based on deep reinforcement learning for a hybrid battery system in electric vehicles consisting of a high-energy and a high-power battery pack. The energy management strategy of the hybrid battery system was developed based on the electrical and thermal characterization of the battery cells, aiming at minimizing the energy loss and increasing both the electrical and thermal safety level of the whole system. Primarily, we designed a novel reward term to explore the optimal operating range of the high-power pack without imposing a rigid constraint of state of charge. Furthermore, various load profiles were randomly combined to train the deep Q-learning model, which avoided the overfitting problem. The training and validation results showed both the effectiveness and reliability of the proposed strategy in loss reduction and safety enhancement. The proposed energy management strategy has demonstrated its superiority over the reinforcement learning-based methods in both computation time and energy loss reduction of the hybrid battery system, highlighting the use of such an approach in future energy management systems. Graphical abstract: Highlights: Energy management of hybrid battery systems in electric vehicles with deep Q-learning. Minimization of the energy loss and increase of both electrical and thermal safety. Exploration of high-power pack's operation range without a rigid SoC constraint. Electrical and thermal characterization of high-energy and high-power batteries. Comparative study with a reinforcement learning-based energy management strategy.