Curriculum-based reinforcement learning for path tracking in an underactuated nonholonomic system. Issue 37 (2022)

• Record Type: Journal Article
• Authors: Chivkula, Prashanth; Rodwell, Colin; Tallapragada, Phanindra
• Is Part Of: IFAC-PapersOnLine. Volume 55:Issue 37(2022)
• Page Start: 339
• Page End: 344
• Publication Date: 2022
• Subjects: Nonholonomic Systems -- Reinforcement Learning -- Path Tracking; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2022.11.207
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: Underactuated mechanical systems with nonholonomic constraints find applications in bioinspired robotics, such as snake-like robots and more recently in fish-like aquatic robots. Animal locomotion suggests that in such bioinspired robots, gaits or cyclic changes in kinematics or shape variables lead to efficient and agile motion. Path tracking in such nonholonomic systems that are not purely kinematic can be a challenging problem. In this paper we consider the problem of path tracking by a modified Chaplygin sleigh with a 'tail' which is a four degree of freedom nonholonomic system, possessing a single internal reaction wheel as an actuator. We develop a curriculum based deep Reinforcement Learning (RL) optimal control approach for simultaneous velocity and path tracking for this system. The curriculum based learning approach first leads to a policy for optimal tracking of limit cycles in a reduced velocity space and then in a next step to track a path. This curriculum approach allows an RL agent to learn the 'mechanics on invariant manifolds' of the system and can be a useful approach in the motion control of high degree of freedom robots with increasing model complexity.