Perturbation analysis of inhomogeneous finite Markov chains. (March 2016)

• Record Type: Journal Article
• Authors: Heidergott, Bernd; Leahu, Haralambie; Löpker, Andreas; Pflug, Georg
• Is Part Of: Advances in applied probability. Volume 48:Number 1(2016)
• Page Start: 255
• Page End: 273
• Publication Date: 2016-03
• Subjects: Finite Markov process, -- inhomogeneous Markov process, -- sensitivity analysis, -- infinitesimal generator; Primary 60J27, -- Secondary 65C05; Probabilities -- Periodicals; Stochastic models -- Periodicals; Electronic journals; Periodicals; 519.2
• DOI: 10.1017/apr.2015.16
• Languages: English
• ISSNs: 0001-8678

Abstract

Abstract: In this paper we provide a perturbation analysis of finite time-inhomogeneous Markov processes. We derive closed-form representations for the derivative of the transition probability at time t, with t > 0. Elaborating on this result, we derive simple gradient estimators for transient performance characteristics either taken at some fixed point in time t, or for the integrated performance over a time interval [0, t ]. Bounds for transient performance sensitivities are presented as well. Eventually, we identify a structural property of the derivative of the generator matrix of a Markov chain that leads to a significant simplification of the estimators.