Deep reinforcement learning enabled self-learning control for energy efficient driving. (February 2019)

• Record Type: Journal Article
• Authors: Qi, Xuewei; Luo, Yadan; Wu, Guoyuan; Boriboonsomsin, Kanok; Barth, Matthew
• Is Part Of: Transportation research. Volume 99(2019)
• Page Start: 67
• Page End: 81
• Publication Date: 2019-02
• Subjects: Deep reinforcement learning -- PHEV -- Energy efficiency -- Self-learning; Transportation -- Periodicals; Transportation -- Technological innovations -- Periodicals; 388.011
• DOI: 10.1016/j.trc.2018.12.018
• Languages: English
• ISSNs: 0968-090X

Abstract

Highlights: CAVs are capable of learning the optimal control strategy that reduce fuel consumption. More than 16% fuel efficiency improvement can be achieved by a deep-reinforcement learning. A dueling neural network structure (DDQN) can further increase the learning speed of the self-cleaning control model. Abstract: To address the air pollution problems and reduce greenhouse gas emissions (GHG), plug-in hybrid electric vehicles (PHEV) have been developed to achieve higher fuel efficiency. The Energy Management System (EMS) is a very important component of a PHEV in achieving better fuel economy and it is a very active research area. So far, most of the existing EMS strategies are either just simply following predefined rules that are not adaptive to changing driving conditions; or heavily relying on accurate prediction of future traffic conditions. Deep learning algorithms have been successfully applied to many complex problems and proved to even outperform human beings in some tasks (e.g., play chess) in recent years, which shows the great potential of such methods in practical engineering problems. In this study, a deep reinforcement learning (Deep Q-network or DQN) based PHEV energy management system is designed to autonomously learn the optimal fuel/electricity splits from interactions between the vehicle and the traffic environment. It is a fully data-driven and self-learning model that does not rely on any prediction, predefined rules or even prior human knowledge. The experiment results show that the proposed model is capable of achieving 16.3% energy savings (with the designed PHEV simulation model) on a typical commute trip, compared to conventional binary control strategies. In addition, a dueling Deep Q-network with dueling structure (DDQN) is also implemented and compared with single DQN in particular with respect to the convergence rate in the training process.