Adaptive constraint satisfaction for Markov decision process congestion games: Application to transportation networks. (May 2023)

• Record Type: Journal Article
• Authors: Li, Sarah H.Q.; Yu, Yue; Miguel, Nicolas I.; Calderone, Dan; Ratliff, Lillian J.; Açıkmeşe, Behçet
• Is Part Of: Automatica. Volume 151(2023)
• Publication Date: 2023-05
• Subjects: Markov decision process -- Incentive design -- Congestion games -- Online optimization -- Transportation systems -- Stochastic games; Automatic control -- Periodicals; Automation -- Periodicals; 629.805
• DOI: 10.1016/j.automatica.2023.110879
• Languages: English
• ISSNs: 0005-1098

Abstract

Abstract: Under the Markov decision process (MDP) congestion game framework, we study the problem of enforcing population distribution constraints on a population of players with stochastic dynamics and coupled congestion costs. Existing research demonstrates that the constraints on the players' population distribution can be satisfied by enforcing tolls. However, computing the minimum toll value for constraint satisfaction requires accurate modeling of the player's congestion costs. Motivated by settings where an accurate congestion cost model may be unavailable (e.g. transportation networks), we consider an MDP congestion game with unknown congestion costs. We assume that a constraint-enforcing authority can repeatedly enforce tolls on a population of players that converges to an ϵ -optimal population distribution for any given toll. We then construct a myopic update algorithm to compute the minimum toll value while ensuring that the constraints are satisfied on average. We analyze how the players' sub-optimal responses to tolls impact the rates of convergence towards the minimum toll value and constraint satisfaction. Finally, we construct a congestion game model for Uber drivers in Manhattan, New York City (NYC) using data from the Taxi and Limousine Commission (TLC) to illustrate how to efficiently reduce congestion while minimizing the impact on driver earnings.