Deep reinforcement learning-based safe interaction for industrial human-robot collaboration using intrinsic reward function. (August 2021)

• Record Type: Journal Article
• Authors: Liu, Quan; Liu, Zhihao; Xiong, Bo; Xu, Wenjun; Liu, Yang
• Is Part Of: Advanced engineering informatics. Volume 49(2021)
• Publication Date: 2021-08
• Subjects: Industrial human-robot collaboration -- Collision avoidance -- Deep reinforcement learning -- Intrinsic reward function; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2021.101360
• Languages: English
• ISSNs: 1474-0346

Abstract

Highlights: A method for safe interaction in industrial human-robot collaboration is proposed. A specific reward function with intrinsic reward and extrinsic reward is designed. Safety and task are considered simultaneously. It contributes to safe human-robot collaboration and reward shaping. Abstract: Aiming at human-robot collaboration in manufacturing, the operator's safety is the primary issue during the manufacturing operations. This paper presents a deep reinforcement learning approach to realize the real-time collision-free motion planning of an industrial robot for human-robot collaboration. Firstly, the safe human-robot collaboration manufacturing problem is formulated into a Markov decision process, and the mathematical expression of the reward function design problem is given. The goal is that the robot can autonomously learn a policy to reduce the accumulated risk and assure the task completion time during human-robot collaboration. To transform our optimization object into a reward function to guide the robot to learn the expected behaviour, a reward function optimizing approach based on the deterministic policy gradient is proposed to learn a parameterized intrinsic reward function. The reward function for the agent to learn the policy is the sum of the intrinsic reward function and the extrinsic reward function. Then, a deep reinforcement learning algorithm intrinsic reward-deep deterministic policy gradient (IRDDPG), which is the combination of the DDPG algorithm and the reward function optimizing approach, is proposed to learn the expected collision avoidance policy. Finally, the proposed algorithm is tested in a simulation environment, and the results show that the industrial robot can learn the expected policy to achieve the safety assurance for industrial human-robot collaboration without missing the original target. Moreover, the reward function optimizing approach can help make up for the designed reward function and improve policy performance.