Learning‐based T‐sHDP(λ) for optimal control of a class of nonlinear discrete‐time systems. (24th October 2021)

• Record Type: Journal Article
• Authors: Yu, Luyang; Liu, Weibo; Liu, Yurong; Alsaadi, Fawaz E.
• Other Names: Karimi Hamid Reza guestEditor.; Wang Ning guestEditor.; Man Zhihong guestEditor.
• Is Part Of: International journal of robust and nonlinear control. Volume 32:Number 5(2022)
• Page Start: 2624
• Page End: 2643
• Publication Date: 2021-10-24
• Subjects: eligibility traces (ET) -- heuristic dynamic programming (HDP) -- learning‐based optimal control -- value iteration; Automatic control -- Periodicals; Control theory -- Periodicals; Nonlinear systems -- Periodicals; 629.836
• DOI: 10.1002/rnc.5847
• Languages: English
• ISSNs: 1049-8923

Abstract

Abstract: This article investigates the optimal control problem via reinforcement learning for a class of nonlinear discrete‐time systems. The nonlinear system under consideration is assumed to be partially unknown. A new learning‐based algorithm, T ‐step heuristic dynamic programming with eligibility traces ( T ‐sHDP( λ )), is proposed to tackle the optimal control problem for such partially unknown system. First, the concerned optimal control problem is turned into its equivalence problem, that is, solving a Bellman equation. Then, the T ‐sHDP( λ ) is utilized to get an approximate solution of Bellman equation, and a rigorous convergence analysis is also conducted as well. Instead of the commonly used single step update approach, the T ‐sHDP( λ ) stores finite step past returns by introducing a parameter, and then utilizes these knowledge to update the value function (VF) of multiple moments synchronously, so as to achieve higher convergence speed. For implementation of T ‐sHDP( λ ), a neural network‐based actor‐critic architecture is applied to approximate VF and optimal control scheme. Finally, the feasibility of the algorithm is demonstrated by two illustrative simulation examples.