Using reinforcement learning techniques to solve continuous‐time non‐linear optimal tracking problem without system dynamics. Issue 12 (1st August 2016)

• Record Type: Journal Article
• Authors: Zhu, Yuanheng; Zhao, Dongbin; Li, Xiangjun
• Is Part Of: IET control theory & applications. Volume 10:Issue 12(2016)
• Page Start: 1339
• Page End: 1347
• Publication Date: 2016-08-01
• Subjects: nonlinear control systems -- continuous time systems -- learning (artificial intelligence) -- optimal control -- dynamic programming -- Lyapunov methods -- linear systems; reinforcement learning -- continuous‐time problem -- nonlinear optimal tracking problem -- adaptive dynamic programming -- model‐free adaptive optimal tracking algorithm -- Lyapunov analysis -- linear system; Control theory -- Periodicals; Automatic control -- Periodicals; 629.8312
• DOI: 10.1049/iet-cta.2015.0769
• Languages: English
• ISSNs: 1751-8644

Abstract

Abstract : The optimal tracking of non‐linear systems without knowing system dynamics is an important and intractable problem. Based on the framework of reinforcement learning (RL) and adaptive dynamic programming, a model‐free adaptive optimal tracking algorithm is proposed in this study. After constructing an augmented system with the tracking errors and the reference states, the tracking problem is converted to a regulation problem with respect to the new system. Several RL techniques are synthesised to form a novel algorithm which learns the optimal solution online in real time without any information of the system dynamics. Continuous adaptation laws are defined by the current observations and the past experience. The convergence is guaranteed by Lyapunov analysis. Two simulations on a linear and a non‐linear systems demonstrate the performance of the proposed approach.