Improved off‐policy reinforcement learning algorithm for robust control of unmodeled nonlinear system with asymmetric state constraints. (25th October 2022)

• Record Type: Journal Article
• Authors: Zhang, Yong; Mu, Chaoxu; Feng, Yanghe; Zhao, Zhijia
• Is Part Of: International journal of robust and nonlinear control. Volume 33:Number 3(2023)
• Page Start: 1607
• Page End: 1632
• Publication Date: 2022-10-25
• Subjects: asymmetric state constraints -- neural networks -- principal component analysis -- reinforcement learning -- robust control; Automatic control -- Periodicals; Control theory -- Periodicals; Nonlinear systems -- Periodicals; 629.836
• DOI: 10.1002/rnc.6432
• Languages: English
• ISSNs: 1049-8923

Abstract

Abstract: In this article, an improved data‐based off‐policy reinforcement learning algorithm is proposed for the robust control of unmodeled nonlinear systems with asymmetric state constraints. An improved nonlinear mapping is defined for the asymmetric state constraint problem, which can ensure that the mapping state has better response speed and amplitude than the original state. Then, an auxiliary mapping error system is constructed for the off‐policy robust controller design. At the same time, an innovative network dimensionality reduction method based on principal component analysis is proposed to simplify the useless activation function of action network in off‐policy algorithm, which can effectively reduce the computational burden of data episodes. Considering the uncertain data caused by disturbances, a dominant data sampling method is designed to extract samples that are beneficial to algorithm convergence. On this basis, the improved off‐policy robust control algorithm is constructed. Based on an industrial manipulator system, the effectiveness of the dominant data sampling method and the improved off‐policy robust control algorithm is verified by comparative simulation.