A general motion controller based on deep reinforcement learning for an autonomous underwater vehicle with unknown disturbances. (January 2023)

• Record Type: Journal Article
• Authors: Huang, Fei; Xu, Jian; Wu, Di; Cui, Yunfei; Yan, Zheping; Xing, Wen; Zhang, Xun
• Is Part Of: Engineering applications of artificial intelligence. Volume 117:Part A(2023)
• Publication Date: 2023-01
• Subjects: Autonomous underwater vehicle -- Deep reinforcement learning -- Motion control -- Maximum a posteriori policy optimization -- Extended state observers; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2022.105589
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: This paper studies the application of deep Reinforcement Learning (RL) in the motion control of an underactuated autonomous underwater vehicle (AUV) with unknown disturbances. Firstly, a general state space, action space and reward function are designed for motion control problems rather than each specific motion control task, which ensures the generality of our method. Furthermore, a virtual AUV model with partial random disturbances is established, and on this basis, a simulation training method is developed to solve the problems of extremely high risk and extremely low efficiency caused by training in actual experiments. Then, in order to directly deploy the optimal control policy obtained through simulation training to an actual AUV, we employ Extended State Observers (ESOs) to estimate the unknown disturbances in five degrees of freedom, and give a deployment method using the estimated values as the disturbance state vector and compensation vector. Combining the above training method and deployment method, a novel general motion controller is proposed. Finally, four different AUV motion control simulations are carried out, and the results confirm the generality and effectiveness of our proposed controller.