MASS autonomous navigation system based on AIS big data with dueling deep Q networks prioritized replay reinforcement learning. (1st April 2022)

• Record Type: Journal Article
• Authors: Gao, Miao; Kang, Zhen; Zhang, Anmin; Liu, Jingxian; Zhao, Fenglong
• Is Part Of: Ocean engineering. Volume 249(2022)
• Publication Date: 2022-04-01
• Subjects: Autonomous navigation system (ANS) -- Marine autonomous surface ship (MASS) -- Dueling-DQNPR -- Reinforcement learning (RL) -- AIS big Data; Ocean engineering -- Periodicals; Ocean engineering; Periodicals; 620.4162
• DOI: 10.1016/j.oceaneng.2022.110834
• Languages: English
• ISSNs: 0029-8018

Abstract

Abstract: Ship autonomous navigation constitutes the most critical step in intelligent ships and is the major prerequisite for the all task completion of the marine autonomous surface ship (MASS). Reinforcement learning (RL) has been widely used in unmanned transport vehicles because of its excellent performance in solving continuous handling problems. This study proposes a MASS autonomous navigation system using dueling deep Q networks prioritized replay (Dueling-DQNPR) based on the ship automatic identification system (AIS) big data. A navigation environment with three difficulty levels were established to train the Dueling-DQNPR network in sequence by setting a reward mechanism. Moreover, the Dueling-DQNPR was improved by combining the prioritized experience replay, dueling structure and long–short-term memory unit to increase the network depth and ability to process continuous data. Finally, simulation training for the AIS trajectory data was carried out in waters near Zhoushan port. The results demonstrated that, through trial and error, the MASS could be controlled to arrive at its destination without collision. As a result, the proposed method may be applicable to MASS on-duty technology and intelligent ships. Highlights: Proposes a MASS ANS using Dueling-DQNPR RL based on the MMG model, ship AIS big data, ECDIS, wind data and current data. Combining dueling structure, Sum tree structure and LSTM cell with deep Q networks to extend dueling deep Q networks prioritized replay into deep neural networks, enhancing the stability and memory of the entire model Attempt to set the maritime navigation regulations for MASS. Adding the new requirement in Rule 18 "Under the maritime encounter mode of unmanned to manned, the level of unmanned ship is lower than manned." And this study assumes that MASS does not apply to Rule 16 in COLREGS.