A reinforcement learning-based near-optimal hierarchical approach for motion control: Design and experiment. (October 2022)

• Record Type: Journal Article
• Authors: Qin, Zhi-Chang; Zhu, Hai-Tao; Wang, Shou-Jun; Xin, Ying; Sun, Jian-Qiao
• Is Part Of: ISA transactions. Volume 129(2022)Part B
• Page Start: 673
• Page End: 683
• Publication Date: 2022-10
• Subjects: Motion control -- Pre-stabilized mechanism -- Model-free optimal control -- Reinforcement learning -- Hierarchical control; Engineering instruments -- Periodicals; Engineering instruments; Periodicals; Electronic journals; 629.805
• DOI: 10.1016/j.isatra.2022.02.034
• Languages: English
• ISSNs: 0019-0578

Abstract

Abstract: As a data-driven design method, model-free optimal control based on reinforcement learning provides an effective way to find optimal control strategies. The design of model-free optimal control is sensitive to system data because it relies on data rather than detailed dynamic models. A prerequisite for generating applicable data is that the system must be open-loop stable (with a stable equilibrium point), which restricts the data-based control design methods in actual control problems and leads to rare experimental studies or verification in the literature. To improve this situation and enrich its applications, we propose a pre-stabilized mechanism and apply it to the motion control of a mechanical system together with a reinforcement learning-based model-free optimal control method, which constitutes a so-called hierarchical control structure. We design two real-time control experiments on an underactuated system to verify its effectiveness. The control results show that the proposed hierarchical control is quite promising in controlling this mechanical system, even though it is open-loop unstable with unknown dynamics. Highlights: Establish a pre-stabilized mechanism to effectively collect training data. The parameters of near-optimal hierarchical control are tuned by system data. Find parameter q is insensitive to the training result of weight coefficients of NNs.