Asynchronous learning for actor–critic neural networks and synchronous triggering for multiplayer system. (October 2022)

• Record Type: Journal Article
• Authors: Wang, Ke; Mu, Chaoxu
• Is Part Of: ISA transactions. Volume 129(2022)Part B
• Page Start: 295
• Page End: 308
• Publication Date: 2022-10
• Subjects: Nonzero-sum differential game -- Neural network -- Actor–critic -- Asynchronous learning -- Synchronous triggering -- Event-triggered communication; Engineering instruments -- Periodicals; Engineering instruments; Periodicals; Electronic journals; 629.805
• DOI: 10.1016/j.isatra.2022.02.007
• Languages: English
• ISSNs: 0019-0578

Abstract

Abstract: In this paper, based on actor–critic neural network structure and reinforcement learning scheme, a novel asynchronous learning algorithm with event communication is developed, so as to solve Nash equilibrium of multiplayer nonzero-sum differential game in an adaptive fashion. From the point of optimal control view, each player or local controller wants to minimize the individual infinite-time cost function by finding an optimal policy. In this novel learning framework, each player consists of one critic and one actor, and implements distributed asynchronous policy iteration to optimize decision-making process. In addition, communication burden between the system and players is effectively reduced by setting up a central event generator. Critic network executes fast updates by gradient-descent adaption while actor network gives event-induced updates using the gradient projection. The closed-loop asymptotic stability is ensured along with uniform ultimate convergence. Then, the effectiveness of the proposed algorithm is substantiated on a four-player nonlinear system, revealing that it can significantly reduce sampling numbers without impairing learning accuracy. Finally, by leveraging nonzero-sum game idea, the proposed learning scheme is also applied to solve the lateral-directional stability of a linear aircraft system, and is further extended to a nonlinear vehicle system for achieving adaptive cruise control. Highlights: A novel asynchronous learning algorithm is developed using actor–critic neural networks. Nash equilibrium of nonlinear multiplayer differential game can be obtained upon event communication. A four-player numerical system demonstrates the effectiveness of the event-based learning algorithm. The proposed learning scheme is also examined on a linear aircraft system and a nonlinear vehicle system.