A general motion control architecture for an autonomous underwater vehicle with actuator faults and unknown disturbances through deep reinforcement learning. (1st November 2022)

• Record Type: Journal Article
• Authors: Huang, Fei; Xu, Jian; Yin, Liangang; Wu, Di; Cui, Yunfei; Yan, Zheping; Chen, Tao
• Is Part Of: Ocean engineering. Volume 263(2022)
• Publication Date: 2022-11-01
• Subjects: Autonomous underwater vehicle -- Deep reinforcement learning -- Motion control -- Actuator faults -- Unknown disturbances; Ocean engineering -- Periodicals; Ocean engineering; Periodicals; 620.4162
• DOI: 10.1016/j.oceaneng.2022.112424
• Languages: English
• ISSNs: 0029-8018

Abstract

Abstract: This paper studies the application of deep reinforcement learning (RL) in the motion control tasks of an underactuated AUV with actuator faults and unknown disturbances. Firstly, a general state space and action space that can be applied to a variety of motion control tasks are customized. Furthermore, a universal reward function is designed to minimize the energy consumption of the AUV under the condition that the learned optimal control policy can achieve fast and high-precision control. Then, a simulation method with random reference values and simulated random disturbances is given to train the deep RL algorithm. In order to deploy the optimal control policy obtained from the simulation training directly to an actual AUV, we design five extended state observers (ESOs) to estimate the unknown disturbances in different directions, and take the estimated values as the disturbance states required to obtain the optimal control action. Combined with these ideas, a general AUV motion control architecture with simulation training and deployment process is developed. Finally, four different AUV motion control experiments are carried out, and the results confirm the generality and effectiveness of our architecture. Highlights: A general motion control architecture for an underactuated autonomous underwater vehicle (AUV) with actuator faults and unknown disturbances based on deep reinforcement learning (RL) is proposed. A general state space, action space and reward function for various motion control tasks are customized. A simulation method with random reference values and simulated random disturbances is given to train the deep RL algorithm in the proposed architecture. Combined with the designed five extended state observers (ESOs), the optimal control policy obtained from simulation training can be directly deployed to an actual AUV.