Motion control of unmanned underwater vehicles via deep imitation reinforcement learning algorithm. Issue 7 (1st May 2020)

• Record Type: Journal Article
• Authors: Chu, Zhenzhong; Sun, Bo; Zhu, Daqi; Zhang, Mingjun; Luo, Chaomin
• Is Part Of: IET intelligent transport systems. Volume 14:Issue 7(2020)
• Page Start: 764
• Page End: 774
• Publication Date: 2020-05-01
• Subjects: neurocontrollers -- intelligent robots -- learning (artificial intelligence) -- three‐term control -- closed loop systems -- motion control -- remotely operated vehicles -- gradient methods -- autonomous underwater vehicles -- control engineering computing; training process -- IL‐TD3 algorithm -- DDPG -- control performance -- PID control -- unmanned underwater vehiclesViadeep imitation reinforcement learning algorithm -- motion control algorithm -- deep imitation reinforcement learning -- unmanned underwater vehicles -- imitation learning twin -- deep deterministic policy gradient -- supervised learning method -- closed‐loop control data -- deep reinforcement learning -- actor–critic architecture -- actor part -- critic part -- current control strategy -- UUV motion control tasks; Intelligent transportation systems -- Periodicals; Electronics in transportation -- Periodicals; 388.31205
• DOI: 10.1049/iet-its.2019.0273
• Languages: English
• ISSNs: 1751-956X

Abstract

Abstract : In this study, a motion control algorithm based on deep imitation reinforcement learning is proposed for the unmanned underwater vehicles (UUVs). The algorithm is called imitation learning (IL) twin delay deep deterministic policy gradient (DDPG) (TD3). It combines IL with DDPG (TD3). In order to accelerate the training process of reinforcement learning, the supervised learning method is used in IL for behaviour cloning from the closed‐loop control data. The deep reinforcement learning employs actor–critic architecture. The actor part executes the control strategy and the critic part evaluates current control strategy. The training efficiency of IL‐TD3 is compared with DDPG and TD3. The simulation results show that the training results of IL‐TD3 converge faster and the training process is more stable than both of them, the convergence rate of IL‐TD3 algorithm during training is about double that of DDPG and TD3. The control performance via IL‐TD3 is superior to PID in UUVs motion control tasks. The average track error of IL‐TD3 is reduced by 70 % than PID control. The average tracking error under thruster fault is almost the same as under normal condition.