A computational study of optimal control of Markov jump systems. Issue 8 (May 2022)

• Record Type: Journal Article
• Authors: Jin, Liqiang; Yin, Yanyan; Lin, Qun; Loxton, Ryan; Teo, Kok Lay
• Is Part Of: Journal of the Franklin Institute. Volume 359:Issue 8(2022)
• Page Start: 3575
• Page End: 3596
• Publication Date: 2022-05
• Subjects: Science -- Periodicals; Technology -- Periodicals; Patents -- United States -- Periodicals; 505
• DOI: 10.1016/j.jfranklin.2022.03.013
• Languages: English
• ISSNs: 0016-0032

Abstract

Abstract: This paper considers a class of optimal control problems governed by Markov jump systems. Our focus is to develop a computational method, based on the control parametrization approach, for solving this class of optimal control problems. Due to the existence of the continuous-time Markov chain, the optimal control problem under consideration is a stochastic optimal control problem, and hence the control parametrization technique cannot be applied directly. For this, a derandomization approach is introduced to obtain a representative deterministic optimal control problem. Then, the control parametrization method is applied to obtain an approximate finite dimensional optimization problem which can be computed numerically using the gradient-based optimization method. For this, the gradient formulas of the cost function and the constraint functions with respect to control variables are derived. Finally, a practical application involving a RLC circuit model is solved using the method proposed.