A numerical simulation of target-directed swimming for a three-link bionic fish with deep reinforcement learning. (June 2023)

• Record Type: Journal Article
• Authors: Zhu, Yi; Pang, Jian-Hua
• Other Names: Tian Fang-Bao guest-editor.; Sun Dongke guest-editor.; Lin Chuandong guest-editor.
• Is Part Of: Proceedings of the Institution of Mechanical Engineers. Volume 237:Number 11(2023)
• Page Start: 2450
• Page End: 2460
• Publication Date: 2023-06
• Subjects: immersed boundary–lattice Boltzmann method -- deep recurrent Q-network -- reinforcement learning -- target-directed swimming -- three-link fish -- bionic fish; Mechanical engineering -- Periodicals; 621.05
• DOI: 10.1177/09544062221079693
• Languages: English
• ISSNs: 0954-4062

Abstract

An accurate and robust feedback control system is of great importance for bionic robotic fish to perform complex tasks. This work presents a numerical study of target-directed swimming for a three-link bionic fish with a feedback control system based on deep reinforcement learning (DRL). The simulation is achieved by using a hybrid method of the DRL method and the immersed boundary–lattice Boltzmann method (IB–LBM). This framework makes use of the high computational efficiency of the IB-LBM for the generation of massive motion data needed in the DRL training. The fish is first trained to swim towards a static target from random orientation and distance. The only information available to the fish is its orientation and distance from the target. It learns an accurate and subtle control policy after the training. Then the control policy is applied to a moving target. Even though the fish encounters some situations that never happened in the training, it can choose appropriate actions to follow the target in close proximity and automatically adapt its velocity with the velocity of the target. Those simulations demonstrate the accuracy, robustness of the control method, and its ability to adapt to new situations.