Output‐feedback Q‐learning for discrete‐time linear H∞ tracking control: A Stackelberg game approach. (18th May 2022)

• Record Type: Journal Article
• Authors: Ren, Yunxiao; Wang, Qishao; Duan, Zhisheng
• Is Part Of: International journal of robust and nonlinear control. Volume 32:Number 12(2022)
• Page Start: 6805
• Page End: 6828
• Publication Date: 2022-05-18
• Subjects: H∞$$ {H}_{\infty } $$ tracking control -- output feedback control -- reinforcement learning -- Stackelberg game; Automatic control -- Periodicals; Control theory -- Periodicals; Nonlinear systems -- Periodicals; 629.836
• DOI: 10.1002/rnc.6169
• Languages: English
• ISSNs: 1049-8923

Abstract

Abstract: In this article, an output‐feedback Q‐learning algorithm is proposed for the discrete‐time linear system to deal with the H ∞ $$ {H}_{\infty } $$ tracking control problem. The problem is formulated as a zero‐sum game in the Stackelberg game framework with a discount factor to make the value function bounded. According to the principle of optimality, the game algebraic Riccati equation (GARE) is derived and solved by the Q‐learning algorithm to get the optimal solution of the Stackelberg game without requiring the knowledge of system dynamics and state. It is proved that the solution of the algorithm converges to the optimal control input and the worst‐case disturbance with excitation noises during training, and the Stackelberg strategy can achieve a lower L 2 $$ {L}_2 $$ disturbance attenuation level than the Nash one. Moreover, the impacts of the discount factor on the stability of the closed‐loop system and solvability of the GARE are analyzed to provide some criteria for the choice of the discount factor. Simulation examples are provided to validate the effectiveness of the algorithm.