A computational study of optimal control of Markov jump systems. Issue 8 (May 2022) - Jin, Liqiang; Yin, Yanyan; Lin, Qun; Loxton, Ryan; Teo, Kok Lay
Abstract: This paper considers a class of optimal control problems governed by Markov jump systems. Our focus is to develop a computational method, based on the control parametrization approach, for solving this class of optimal control problems. Due to the existence of the continuous-time Markov chain, the optimal control problem under consideration is a stochastic optimal control problem, and hence the control parametrization technique cannot be applied directly. For this, a derandomization approach is introduced to obtain a representative deterministic optimal control problem. Then, the control parametrization method is applied to obtain an approximate finite dimensional optimization problem which can be computed numerically using the gradient-based optimization method. For this, the gradient formulas of the cost function and the constraint functions with respect to control variables are derived. Finally, a practical application involving a RLC circuit model is solved using the method proposed.