Imitation of Real Lane-Change Decisions Using Reinforcement Learning. Issue 2 (2021)

• Record Type: Journal Article
• Authors: Zhao, Lu; Farhi, Nadir; Christoforou, Zoi; Haddadou, Nadia
• Is Part Of: IFAC-PapersOnLine. Volume 54:Issue 2(2021)
• Page Start: 203
• Page End: 209
• Publication Date: 2021
• Subjects: Traffic Models -- Artificial intelligence in transportation -- lane-change model -- reinforcement learning -- human driving behavior; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2021.06.023
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: Microscopic modeling of human driving consists generally in combining both car-following and lane-change models. While the human car-following process has been extensively developed and well modeled, the lane-change behavior is more complex to understand and still remains to be explored. Classical lane-change models are usually rule-based and handcrafted, that tend to exhibit limited performance. Machine Learning algorithms, particularly Reinforcement Learning (RL) ones, provide an alternative approach and have recently achieved high success in modeling difficult decision-making processes in many fields. We propose in this article a reinforcement learning based model for the human lane-change behavior, with an online calibration of real lane-change decisions, extracted from the NGSIM data-set. In addition, we use the traffic vehicular simulator SUMO ("Simulation of Urban Mobility") to create a numerical simulation environment. The utilization of numerical traffic simulation allows us enriching the data-set, for training the agent to find an optimal policy for lane change. Thus, about 13% additional traffic situations, not present in the real data, are created by the traffic simulation environment. The trained agent is collision-free and human-like who is satisfactory to real data and also to the additional simulated data. Moreover, our RL model can perform up to 95.37% of the real decisions observed in the data-set.