A human-like collision avoidance method for autonomous ship with attention-based deep reinforcement learning. (15th November 2022)

• Record Type: Journal Article
• Authors: Jiang, Lingling; An, Lanxuan; Zhang, Xinyu; Wang, Chengbo; Wang, Xinjian
• Is Part Of: Ocean engineering. Volume 264(2022)
• Publication Date: 2022-11-15
• Subjects: Maritime safety -- Ship collision avoidance -- Self-attention mechanism -- Deep reinforcement learning -- Autonomous ship; Ocean engineering -- Periodicals; Ocean engineering; Periodicals; 620.4162
• DOI: 10.1016/j.oceaneng.2022.112378
• Languages: English
• ISSNs: 0029-8018

Abstract

Abstract: Reinforcement learning has the characteristics of simple structure and strong adaptability, which has been widely used in the field of ship autonomous collision avoidance. In order to solve the problem of collision avoidance in multi-ship encounter situation, a novel collision avoidance method for autonomous ship with attention-based deep reinforcement learning (ADRL) is proposed, it consists of two parts, risk assessment module and motion planning module, the difference between the former and the existing collision risk calculation method is that from the officer's attention distribution, it encode the ship's information through the local map, and calculate each ship's collision avoidance decision in the form of attention score in real time under the constraints of the COLREGS. In addition, a composite learning method is designed, which integrates supervised learning into the common direct environmental exploration model, which accelerates the exploration efficiency of the model and shows excellent learning performance. Finally, based on the Open AI Gym platform, static obstacle situation, dynamic multi-ship encounter situation, dynamic and static obstacle coexistence situation are designed, and the rationality and effectiveness of collision avoidance decision are analyzed from the perspectives of collision risk and the closest safety distance respectively. Highlights: A novel decision-making model with attention-mechanism based on deep reinforcement learning is proposed, which includes ship risk assessment module and motion planning module. A risk assessment model based on driver's attention distribution is proposed. By encoding the navigation status information, the collision risk is caculated by neural network. A composite collision avoidance decision training method combining supervised learning is designed, it shows efficient learning performance and good applicability conpared with the traditional learning model. The method presented in this study is able to solve collision avoidance decisions in complex navigation situations in real time, varieties of simulations situation are designed to verify the effectiveness of method.