Neural-network-based discounted optimal control via an integrated value iteration with accuracy guarantee. (December 2021)

• Record Type: Journal Article
• Authors: Ha, Mingming; Wang, Ding; Liu, Derong
• Is Part Of: Neural networks. Volume 144(2021)
• Page Start: 176
• Page End: 186
• Publication Date: 2021-12
• Subjects: Adaptive dynamic programming -- Data-based discounted optimal control -- Value iteration -- Lyapunov method -- Uniformly ultimately bounded stability; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2021.08.025
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: A data-based value iteration algorithm with the bidirectional approximation feature is developed for discounted optimal control. The unknown nonlinear system dynamics is first identified by establishing a model neural network. To improve the identification precision, biases are introduced to the model network. The model network with biases is trained by the gradient descent algorithm, where the weights and biases across all layers are updated. The uniform ultimate boundedness stability with a proper learning rate is analyzed, by using the Lyapunov approach. Moreover, an integrated value iteration with the discounted cost is developed to fully guarantee the approximation accuracy of the optimal value function. Then, the effectiveness of the proposed algorithm is demonstrated by carrying out two simulation examples with physical backgrounds.