Adaptive dynamic programming for discrete-time linear quadratic regulation based on multirate generalised policy iteration. Issue 6 (3rd June 2018)

• Record Type: Journal Article
• Authors: Chun, Tae Yoon; Lee, Jae Young; Park, Jin Bae; Choi, Yoon Ho
• Is Part Of: International journal of control. Volume 91:Issue 6(2018)
• Page Start: 1223
• Page End: 1240
• Publication Date: 2018-06-03
• Subjects: Multirate generalised policy iteration -- heuristic dynamic programming -- dual heuristic dynamic programming -- adaptive dynamic programming -- mixed-mode convergence -- linear quadratic regulation; Automatic control -- Periodicals; Electronic journals; 629.8
• DOI: 10.1080/00207179.2017.1312669
• Languages: English
• ISSNs: 0020-7179

Abstract

ABSTRACT: In this paper, we propose two multirate generalised policy iteration (GPI) algorithms applied to discrete-time linear quadratic regulation problems. The proposed algorithms are extensions of the existing GPI algorithm that consists of the approximate policy evaluation and policy improvement steps. The two proposed schemes, named heuristic dynamic programming (HDP) and dual HDP (DHP), based on multirate GPI, use multi-step estimation ( M -step Bellman equation) at the approximate policy evaluation step for estimating the value function and its gradient called costate, respectively. Then, we show that these two methods with the same update horizon can be considered equivalent in the iteration domain. Furthermore, monotonically increasing and decreasing convergences, so called value iteration (VI)-mode and policy iteration (PI)-mode convergences, are proved to hold for the proposed multirate GPIs. Further, general convergence properties in terms of eigenvalues are also studied. The data-driven online implementation methods for the proposed HDP and DHP are demonstrated and finally, we present the results of numerical simulations performed to verify the effectiveness of the proposed methods.