Analysis of averages over distributions of Markov processes. (December 2018)

• Record Type: Journal Article
• Authors: Valenzuela, Patricio E.; Rojas, Cristian R.; Hjalmarsson, Håkan
• Is Part Of: Automatica. Volume 98(2018)
• Page Start: 354
• Page End: 357
• Publication Date: 2018-12
• Subjects: System identification -- Input design -- Markov chains; Automatic control -- Periodicals; Automation -- Periodicals; 629.805
• DOI: 10.1016/j.automatica.2018.09.016
• Languages: English
• ISSNs: 0005-1098

Abstract

Abstract: In problems of optimal control of Markov decision processes and optimal design of experiments, the occupation measure of a Markov process is designed in order to maximize a specific reward function. When the memory of such a process is too long, or the process is non-Markovian but mixing, it makes sense to approximate it by that of a shorter memory Markov process. This note provides a specific bound for the approximation error introduced in these schemes. The derived bound is then applied to the proposed solution of a recently introduced approach to optimal input design for nonlinear systems.