Deep reinforcement learning with combinatorial actions spaces: An application to prescriptive maintenance. (May 2023)

• Record Type: Journal Article
• Authors: Goby, Niklas; Brandt, Tobias; Neumann, Dirk
• Is Part Of: Computers & industrial engineering. Volume 179(2023)
• Publication Date: 2023-05
• Subjects: Maintenance -- Prescriptive analytics -- Deep reinforcement learning -- Combinatorial action space -- Prescriptive maintenance; Engineering -- Data processing -- Periodicals; Industrial engineering -- Periodicals; 620.00285
• DOI: 10.1016/j.cie.2023.109165
• Languages: English
• ISSNs: 0360-8352

Abstract

Abstract: In this paper, we leverage a prescriptive analytics approach based on deep reinforcement learning and adapt it for sequential decision-problems with large, noisy state spaces and combinatorial actions spaces. We implement a novel mechanism that uses deep learning to reduce the action space and apply the approach to the context of maintenance management. We show that our method substantially outperforms established baseline methods from practice and research, closing more than 90 percent of the cost gap between the next-best solution and the optimum under perfect information. In addition to reducing costs, the specifically-designed reward function incentivizes bundling maintenance actions in a way that fully utilizes the available number of workers. Thereby, the number of time steps in which any maintenance action occurs is reduced. This decreases the organizational and operational impact of maintenance in real-world settings as disruptions can be limited to a few days. Beyond this context, our work illustrates the potential of prescriptive approaches based on deep reinforcement learning in other applications that face similarly challenging problem settings. Highlights: We present a prescriptive maintenance approach based on deep reinforcement learning. Adapted for large, noisy state spaces and combinatorial actions spaces. Method outperforms established methods, reducing costs by more than 70 percent. Agent bundles maintenance actions, fully utilizing the available number of workers.