Autonomous navigation at unsignalized intersections: A coupled reinforcement learning and model predictive control approach. (June 2022)

• Record Type: Journal Article
• Authors: Bautista-Montesano, Rolando; Galluzzi, Renato; Ruan, Kangrui; Fu, Yongjie; Di, Xuan
• Is Part Of: Transportation research. Volume 139(2022)
• Publication Date: 2022-06
• Subjects: Urban navigation -- Path planning -- Reinforcement learning -- Model predictive control; Transportation -- Periodicals; Transportation -- Technological innovations -- Periodicals; 388.011
• DOI: 10.1016/j.trc.2022.103662
• Languages: English
• ISSNs: 0968-090X

Abstract

Abstract: This paper develops an integrated safety-enhanced reinforcement learning (RL) and model predictive control (MPC) framework for autonomous vehicles (AVs) to navigate unsignalized intersections. Researchers have extensively studied how AVs drive along highways. Nonetheless, how AVs navigate intersections in urban environments remains a challenging task due to the constant presence of moving road users, including turning vehicles, crossing or jaywalking pedestrians, and cyclists. AVs are thus required to learn and adapt to a dynamically evolving urban traffic environment. This paper proposes a design benchmark that allows AVs to sense the real-time traffic environment and perform path planning. The agent dynamically generates curves for feasible paths. The ego vehicle attempts to follow these paths under specific constraints. RL and MPC navigation algorithms run in parallel and are suitably selected to enhance ego vehicle safety. The ego AV is modeled with lateral and longitudinal dynamics and trained in a T-intersection using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm under various traffic scenarios. It is then tested on a straight road and a single or multi-lane intersections. All these experiments achieve desirable outcomes in terms of crash avoidance, driving efficiency, comfort, and tracking accuracy. The developed AV navigation system provides a design benchmark for an adaptive AV that can navigate unsignalized intersections. Highlights: Vehicle navigation based on RL and MPC algorithms that run in parallel A dynamic trajectory planner generates a set of waypoints A safety-enhanced switch changes from RL to MPC when needed RL methods are trained using a realistic vehicle dynamic model.