Soft formation control for unmanned surface vehicles under environmental disturbance using multi-task reinforcement learning. (15th September 2022)

• Record Type: Journal Article
• Authors: Jin, Kefan; Wang, Jian; Wang, Hongdong; Liang, Xiaofeng; Guo, Yongjin; Wang, Mianjin; Yi, Hong
• Is Part Of: Ocean engineering. Volume 260(2022)
• Publication Date: 2022-09-15
• Subjects: Ocean engineering -- Periodicals; Ocean engineering; Periodicals; 620.4162
• DOI: 10.1016/j.oceaneng.2022.112035
• Languages: English
• ISSNs: 0029-8018

Abstract

Abstract: This paper proposes a distributed soft formation collision avoidance strategy to address the problem of formation obstacle avoidance in complex scenario under the environment interference where unmanned surface vehicles (USVs) are restricted in observation and possess no global reference frame for localisation. A multi-task training framework for formation control is also developed based on the motion characteristics of USVs and the leader–follower method. The soft actor–critic (SAC) reinforcement learning algorithm is adapted to construct agents. With elaborated auxiliary tasks, the soft formation algorithm demonstrates strong resilience to disturbances caused by unknown environmental loads or obstacle avoidance needs with only partial information about the environmental state, thus maintaining the formation shape. The proposed trajectory communication system and policy sharing mechanism allow USVs to approach the destination and avoid collisions whilst maintaining a noncompact formation that can be broken up temporarily to improve obstacle avoidance and can be restored quickly when the obstacle is avoided. Trained agents can be applied to different formations of varying sizes. Simulation results demonstrate the feasibility and effectiveness of the proposed method in different complex sea scenarios and its robustness to unknown environmental disturbances. Highlights: Formation control under harsh conditions with partial observation and dynamic environmental interference is achieved. SFC method can easily adapt to changes in formation shape and size during task procedure. Dynamic formation collision avoidance task is achieved under complex scenario where obstacles are densely located. Complex formation task is decoupled with elaborated multi tasks.