Off-policy reinforcement learning-based novel model-free minmax fault-tolerant tracking control for industrial processes. (July 2022)

• Record Type: Journal Article
• Authors: Li, Xueyu; Luo, Qiuwen; Wang, Limin; Zhang, Ridong; Gao, Furong
• Is Part Of: Journal of process control. Volume 115(2022)
• Page Start: 145
• Page End: 156
• Publication Date: 2022-07
• Subjects: Industrial process -- Off-policy reinforcement learning -- Novel model-free minmax fault-tolerant tracking control -- Actuator failure; Process control -- Periodicals; Fabrication -- Contrôle -- Périodiques; Process control; Periodicals; Electronic journals; 660.281
• DOI: 10.1016/j.jprocont.2022.05.006
• Languages: English
• ISSNs: 0959-1524

Abstract

Abstract: For industrial processes with external disturbance and actuator failure, off-policy reinforcement learning-based novel model-free minmax fault-tolerant control is proposed in this paper to solve H ∞ fault-tolerant tracking control problem. An augmented model equivalent to the original system is constructed, and the state of the new augmented model is composed of state increment and tracking error of the original system. The original H ∞ fault-tolerant tracking problem was transformed into the linear quadratic zero-sum game problem by establishing performance index function, and the Game Algebraic Riccati Equation (GARE) was established. Then Q function was introduced and the Off-policy reinforcement learning algorithm was designed. Different from the traditional model-based fault-tolerant control method, the proposed algorithm does not need the knowledge of system dynamics, and it can learn from the measured data of the system trajectory to solve the GARE. In addition, it is proved that the probing noise added to satisfy the persistent excitation condition does not cause bias. A simulation example of injection molding process is used to verify the effectiveness of the proposed algorithm. Highlights: A new off-policy reinforcement learning algorithm based novel model-free minmax fault-tolerant control is proposed. The optimal control strategy is transformed into minmax zero-sum game method. Off-policy Q-learning algorithm is used to solve the optimal control strategy and the worst external disturbance.