A Sensitivity‐Based Construction Approach to Variance Minimization of Markov Decision Processes. Issue 3 (8th August 2018)

• Record Type: Journal Article
• Authors: Huang, Yonghao; Chen, Xi
• Is Part Of: Asian journal of control. Volume 21:Issue 3(2019)
• Page Start: 1166
• Page End: 1178
• Publication Date: 2018-08-08
• Subjects: Variance minimization -- sensitivity‐based approach -- Markov decision process; Automatic control -- Periodicals; Control theory -- Periodicals; 629.805
• DOI: 10.1002/asjc.1875
• Languages: English
• ISSNs: 1561-8625

Abstract

Abstract: With a long‐run average performance as the primary criterion for a Markov decision process, variance measures are studied as its secondary criteria. The steady‐state variance and the limiting average variance along a sample path are discussed. The latter one is difficult to handle due to its special form. With a sensitivity‐based approach, the difference formula for the sample‐path variance under different policies is intuitively constructed and then the optimality equation is presented. Moreover a policy iteration algorithm is developed. This work extends the sensitivity‐based construction approach to Markov decision processes with non‐standard performance criteria. The difference between these two types of variance and bias criteria is illustrated with a numerical example.