Adaptive optimal output tracking of continuous-time systems via output-feedback-based reinforcement learning. (December 2022)

• Record Type: Journal Article
• Authors: Chen, Ci; Xie, Lihua; Xie, Kan; Lewis, Frank L.; Xie, Shengli
• Is Part Of: Automatica. Volume 146(2022)
• Publication Date: 2022-12
• Subjects: Reinforcement learning -- Off-policy -- Output tracking -- Output feedback -- Adaptive optimal control; Automatic control -- Periodicals; Automation -- Periodicals; 629.805
• DOI: 10.1016/j.automatica.2022.110581
• Languages: English
• ISSNs: 0005-1098

Abstract

Abstract: Reinforcement learning provides a powerful tool for designing a satisfactory controller through interactions with the environment. Although off-policy learning algorithms were recently designed for tracking problems, most of these results either are full-state feedback or have bounded control errors, which may not be flexible or desirable for engineering problems in the real world. To address these problems, we propose an output-feedback-based reinforcement learning approach that allows us to find the optimal control solution using input–output data and ensure the asymptotic tracking control of continuous-time systems. More specifically, we first propose a dynamical controller revised from the standard output regulation theory and use it to formulate an optimal output tracking problem. Then, a state observer is used to re-express the system state. Consequently, we address the rank issue of the parameterization matrix and analyze the state re-expression error that are crucial for transforming the off-policy learning into an output-feedback form. A comprehensive simulation study is given to demonstrate the effectiveness of the proposed approach.