Coupling based estimation approaches for the average reward performance potential in Markov chains. (July 2018)

• Record Type: Journal Article
• Authors: Li, Yanjie; Wu, Xinyu; Lou, Yunjiang; Chen, Haoyao; Li, Jiangang
• Is Part Of: Automatica. Volume 93(2018)
• Page Start: 172
• Page End: 182
• Publication Date: 2018-07
• Subjects: Value function -- Performance potential -- Perturbation realization factor -- Estimation with geometric variance reduction (EGVR) -- Coupling techniques; Automatic control -- Periodicals; Automation -- Periodicals; 629.805
• DOI: 10.1016/j.automatica.2018.03.011
• Languages: English
• ISSNs: 0005-1098

Abstract

Abstract: Performance potential is an important concept in the sensitivity analysis of Markov chains. The estimation of performance potential provides the basis for the simulation-based optimization and sensitivity analysis of Markov chains. In this study, we present novel estimation approaches for the average reward (or cost) performance potential by combining perturbation realization factors and coupling techniques for Markov chains with finite state space. These approaches can effectively implement estimation with geometric variance reduction for average reward performance potential. Meanwhile, a number of coupling methods, including two optimal coupling methods, can be applied to further reduce estimation variance or simulation time. The numerical tests show that our approaches can significantly enhance the simulation efficiency.