Approximating the model of a water distribution network as a Markov decision process. Issue 20 (2022)

• Record Type: Journal Article
• Authors: Misra, Rahul; Wisniewski, Rafał; Kallesøe, Carsten S.
• Is Part Of: IFAC-PapersOnLine. Volume 55:Issue 20(2022)
• Page Start: 271
• Page End: 276
• Publication Date: 2022
• Subjects: Modeling for control optimization -- Markov decision process -- Stochastic differential equations -- Dynamic programming -- Markov chain approximation -- Monte Carlo methods; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2022.09.107
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: In this paper, our objective is to convert the model of a water distribution network described via Stochastic differential equations (SDE) into a Markov decision process (MDP). The motivation behind this work is that while MDP's represents the underlying dynamics for dynamic programming and reinforcement learning, the actual underlying model is best described via differential equations, and therefore, we would like to convert the SDE to MDP. We have applied Kushner's Markov chain approximation (MCA) method and verified it using a novel modified Monte Carlo method which can be considered as an alternative to the well-known Kushner's MCA. Both the methods approximate the value function and simulation studies show that the obtained value functions from both the methods converge to almost the same value.