Direct Adaptive Pole-Placement Controller using Deep Reinforcement Learning: Application to AUV Control⁎This work was supported by ISblue project, Interdisciplinary graduate school for the blue planet (ANR-17-EURE-0015) and co-funded by a grant from the French government under the program "Investissements d'Avenir". Issue 16 (2021)

• Record Type: Journal Article
• Authors: Chaffre, Thomas; Chenadec, Gilles Le; Sammut, Karl; Chauveau, Estelle; Clement, Benoit
• Is Part Of: IFAC-PapersOnLine. Volume 54:Issue 16(2021)
• Page Start: 333
• Page End: 340
• Publication Date: 2021
• Subjects: Adaptive control -- Pole-placement -- Deep reinforcement learning -- Underwater vehicle; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2021.10.113
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: In this paper we investigate a direct adaptive learning-based tuning strategy for the control of an underwater vehicle under unknown disturbances. This process can be seen as a double integrator without delay and is usually regulated using a PD/PID type controller. A trade-off between performance and robustness may be found when tuning their parameters because a single optimal controller for multiple operating condition does not exist. Therefore, we use a re-parametrization of the PID controller gains in a space of poles where controller stability is guaranteed. We propose to use the maximum entropy deep reinforcement learning algorithm called SAC to explore this space. The adaptation procedure is able to capture a great variety of desired pole locations in order to adapt to process variations without measuring them. Simulation outcomes show the advantages of this approach.