Hierarchical reinforcement learning for transportation infrastructure maintenance planning. (July 2023)

• Record Type: Journal Article
• Authors: Hamida, Zachary; Goulet, James-A.
• Is Part Of: Reliability engineering & system safety. Volume 235(2023)
• Publication Date: 2023-07
• Subjects: Maintenance planning -- Reinforcement learning -- RL environment -- Deep Q-learning -- Infrastructure deterioration -- State-space models; Reliability (Engineering) -- Periodicals; System safety -- Periodicals; Industrial safety -- Periodicals; Fiabilité -- Périodiques; Sécurité des systèmes -- Périodiques; Sécurité du travail -- Périodiques; 620.00452
• DOI: 10.1016/j.ress.2023.109214
• Languages: English
• ISSNs: 0951-8320

Abstract

Abstract: Maintenance planning on bridges commonly faces multiple challenges, mainly related to complexity and scale. Those challenges stem from the large number of structural elements in each bridge in addition to the uncertainties surrounding their health condition, which is monitored using visual inspections at the element-level. Recent developments have relied on deep reinforcement learning (RL) for solving maintenance planning problems, with the aim to minimize the long-term costs. Nonetheless, existing RL based solutions have adopted approaches that often lacked the capacity to scale due to the inherently large state and action spaces. The aim of this paper is to introduce a hierarchical RL formulation for maintenance planning, which naturally adapts to the hierarchy of information and decisions in infrastructure. The hierarchical formulation enables decomposing large state and action spaces into smaller ones, by relying on state and temporal abstraction. An additional contribution from this paper is the development of an open-source RL environment that uses state-space models (SSM) to describe the propagation of the deterioration condition and speed over time. The functionality of this new environment is demonstrated by solving maintenance planning problems at the element-level, and the bridge-level. Highlights: Hierarchical reinforcement learning (RL) formulation for bridge maintenance planning. Decision-making based on the deterioration condition and deterioration speed. Maintenance planning at element and bridge-level. Benchmark RL environment for transportation infrastructure maintenance policies.