Unmanned surface vessel obstacle avoidance with prior knowledge‐based reward shaping. (29th December 2020)

• Record Type: Journal Article
• Authors: Wang, Wei; Luo, Xiangfeng; Li, Yang; Xie, Shaorong
• Is Part Of: Concurrency and computation. Volume 33:Number 9(2021)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-12-29
• Subjects: complex dynamic environment -- obstacle avoidance -- prior knowledge -- reinforcement learning -- reward shaping -- unmanned surface vessel; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6110
• Languages: English
• ISSNs: 1532-0626

Abstract

Abstract: Autonomous obstacle avoidance control of unmanned surface vessels (USVs) in complex marine environments is always fundamental for its scientific search and detection. Traditional methods usually model USV motion and environments in a mathematical way that needs perceptual information. Unfortunately, it is difficult to provide sufficient perceptual information due to complex marine environments, resulting in inaccurate modeling. Reinforcement learning has recently enjoyed increasing popularity in the problem of obstacle avoidance since it can settle problems by partially observable environment information. However, the autonomous USV obstacle avoidance using reinforcement learning is still facing the challenge of designing appropriate reward functions under complex marine environments. To address these issues, we propose a prior knowledge‐based USV reinforcement learning obstacle avoidance algorithm. In this algorithm, an actor‐critic network is used as the main architecture of the algorithm, and prior knowledge‐based reward shaping used to design relevant reward function for USV obstacle avoidance. A standard USV based on a visual sensor is designed, and the state input of the algorithm is through USV's front vision sensor. We conducted simulation experiments and results prove that our algorithm can effectively converge, and USV achieves high velocity and low collision rate in the complex marine environment.