Traffic Signal Control based on Markov Decision Process*. Issue 3 (2016)

• Record Type: Journal Article
• Authors: Xu, Yunwen; Xi, Yugeng; Li, Dewei; Zhou, Zhao
• Is Part Of: IFAC-PapersOnLine. Volume 49:Issue 3(2016)
• Page Start: 67
• Page End: 72
• Publication Date: 2016
• Subjects: Markov state transition model -- Traffic signal control -- Policy iteration algorithm -- Markov decision process; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2016.07.012
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: This paper proposes a Morkov state transition model for an isolated intersection in urban traffic and formulates the traffic signal control problem as a Markov Decision Process(MDP). In order to reduce computational burden, a sensitivity-based policy iteration(PI) algorithm is introduced to solve the MDP. The proposed model is stage-varying according to traffic flow variation around the intersection, and the state transition matrices and cost matrices are updated so that a new optimal policy can be searched by the PI algorithm. The proposed model also can be easily extended from an isolated intersection to a traffic network based on the space-time distribution characteristics of traffic flow, so as the PI algorithm. The numerical experiments of a small traffic network show that this approach is capable of reducing the number of vehicles substantially compared with the fixed-time control particularly for high traffic demand, while being computationally efficient.