Automated vehicle's behavior decision making using deep reinforcement learning and high-fidelity simulation environment. (October 2019)

• Record Type: Journal Article
• Authors: Ye, Yingjun; Zhang, Xiaohui; Sun, Jian
• Is Part Of: Transportation research. Volume 107(2019)
• Page Start: 155
• Page End: 170
• Publication Date: 2019-10
• Subjects: Automated vehicle -- Decision making -- Deep reinforcement learning -- Reward function; Transportation -- Periodicals; Transportation -- Technological innovations -- Periodicals; 388.011
• DOI: 10.1016/j.trc.2019.08.011
• Languages: English
• ISSNs: 0968-090X

Abstract

Highlights: A novel training platform for automated vehicle (AV)'s decision-making is proposed. Deep deterministic policy gradient algorithm is adopted for continuous state decision of AV. Details about reward setting and accelerated methods for DRL training are investigated. A multi-task driving model integrating CF and LC behavior is trained simultaneously. Abstract: Automated vehicles (AVs) are deemed to be the key element for the intelligent transportation system in the future. Many studies have been made to improve AVs' ability of environment recognition and vehicle control, while the attention paid to decision making is not enough and the existing decision algorithms are very preliminary. Therefore, a framework of the decision-making training and learning is put forward in this paper. It consists of two parts: the deep reinforcement learning (DRL) training program and the high-fidelity virtual simulation environment. Then the basic microscopic behavior, car-following (CF), is trained within this framework. In addition, theoretical analysis and experiments were conducted to evaluate the proposed reward functions for accelerating training using DRL. The results show that on the premise of driving comfort, the efficiency of the trained AV increases 7.9% and 3.8% respectively compared to the classical adaptive cruise control models, intelligent driver model and constant-time headway policy. Moreover, on a more complex three-lane section, we trained an integrated model combining both CF and lane-changing behavior, with the average speed further growing 2.4%. It indicates that our framework is effective for AV's decision-making learning.