Adaptive railway traffic control using approximate dynamic programming. (April 2020)

• Record Type: Journal Article
• Authors: Ghasempour, Taha; Heydecker, Benjamin
• Is Part Of: Transportation research. Volume 113(2020)
• Page Start: 91
• Page End: 107
• Publication Date: 2020-04
• Subjects: Approximate dynamic programming -- Reinforcement learning -- Railway traffic management -- Adaptive control; Transportation -- Periodicals; Transportation -- Technological innovations -- Periodicals; 388.011
• DOI: 10.1016/j.trc.2019.04.002
• Languages: English
• ISSNs: 0968-090X

Abstract

Highlights: Development of an adaptive control system to manage railway traffic in real-time. Application of approximate dynamic programming to railway traffic management. Application of reinforcement learning to approximate railway traffic states. A computationally efficient approach to manage the stochastic nature of railways. Abstract: This study presents an adaptive railway traffic controller for real-time operations based on approximate dynamic programming (ADP). By assessing requirements and opportunities, the controller aims to limit consecutive delays resulting from trains that entered a control area behind schedule by sequencing them at a critical location in a timely manner, thus representing the practical requirements of railway operations. This approach depends on an approximation to the value function of dynamic programming after optimisation from a specified state, which is estimated dynamically from operational experience using reinforcement learning techniques. By using this approximation, the ADP avoids extensive explicit evaluation of performance and so reduces the computational burden substantially. In this investigation, we explore formulations of the approximation function and variants of the learning techniques used to estimate it. Evaluation of the ADP methods in a stochastic simulation environment shows considerable improvements in consecutive delays by comparison with the current industry practice of First-Come-First-Served sequencing. We also found that estimates of parameters of the approximate value function are similar across a range of test scenarios with different mean train entry delays.