Reinforcement learning based on movement primitives for contact tasks. (April 2020)

• Record Type: Journal Article
• Authors: Kim, Young-Loul; Ahn, Kuk-Hyun; Song, Jae-Bok
• Is Part Of: Robotics and computer-integrated manufacturing. Volume 62(2020)
• Publication Date: 2020-04
• Subjects: AI-based methods -- Force control -- Deep Learning in robotics and automation; Robots, Industrial -- Periodicals; Computer integrated manufacturing systems -- Periodicals; Robotics -- Periodicals; Robots industriels -- Périodiques; Productique -- Périodiques; Robotique -- Périodiques; 670.285
• DOI: 10.1016/j.rcim.2019.101863
• Languages: English
• ISSNs: 0736-5845

Abstract

Highlights: For a reinforcement learning method applicable to a force control-based contact task, the main contributions of the proposed method are as follows. A trajectory planner that can learn, utilize various sensor signals, and generate a continuous task trajectory is proposed. An imitation learning algorithm suitable for the proposed trajectory planner is developed. Reinforcement learning is applied to the proposed trajectory planner using DDPG. Abstract: Recently, robot learning through deep reinforcement learning has incorporated various robot tasks through deep neural networks, without using specific control or recognition algorithms. However, this learning method is difficult to apply to the contact tasks of a robot, due to the exertion of excessive force from the random search process of reinforcement learning. Therefore, when applying reinforcement learning to contact tasks, solving the contact problem using an existing force controller is necessary. A neural-network-based movement primitive (NNMP) that generates a continuous trajectory which can be transmitted to the force controller and learned through a deep deterministic policy gradient (DDPG) algorithm is proposed for this study. In addition, an imitation learning algorithm suitable for NNMP is proposed such that the trajectories similar to the demonstration trajectory are stably generated. The performance of the proposed algorithms was verified using a square peg-in-hole assembly task with a tolerance of 0.1 mm. The results confirm that the complicated assembly trajectory can be learned stably through NNMP by the proposed imitation learning algorithm, and that the assembly trajectory is improved by learning the proposed NNMP through the DDPG algorithm.