Learning synergies based in‐hand manipulation with reward shaping. Issue 3 (10th July 2020)

• Record Type: Journal Article
• Authors: Deng, Zhen; Zhang, Jian Wei
• Is Part Of: CAAI transactions on intelligence technology. Volume 5:Issue 3(2020)
• Page Start: 141
• Page End: 149
• Publication Date: 2020-07-10
• Subjects: learning (artificial intelligence) -- manipulators -- dexterous manipulators; multifingered robotic hands -- reward shaping -- in‐hand manipulation learning -- hand synergies -- low‐dimensional hand posture space -- robotic hand postures; Artificial intelligence -- Periodicals; Computer science -- Periodicals; Artificial intelligence; Computer science; Electronic journals; Periodicals; 006.305
• DOI: 10.1049/trit.2019.0094
• Languages: English
• ISSNs: 2468-6557

Abstract

Abstract : In‐hand manipulation is a fundamental ability for multi‐fingered robotic hands that interact with their environments. Owing to the high dimensionality of robotic hands and intermittent contact dynamics, effectively programming a robotic hand for in‐hand manipulations is still a challenging problem. To address this challenge, this work employs deep reinforcement learning (DRL) algorithm to learn in‐hand manipulations for multi‐fingered robotic hands. A reward‐shaping method is proposed to assist the learning of in‐hand manipulation. The synergy of robotic hand postures is analysed to build a low‐dimensional hand posture space. Two additional rewards are designed based on both the analysis of hand synergies and its learning history. The two additional rewards cooperating with an extrinsic reward are used to assist the in‐hand manipulation learning. Three value functions are trained jointly with respect to their reward functions. Then they cooperate to optimise a control policy for in‐hand manipulation. The reward shaping not only improves the exploration efficiency of the DRL algorithm but also provides a way to incorporate domain knowledge. The performance of the proposed learning method is evaluated with object rotation tasks. Experimental results demonstrated that the proposed learning method enables multi‐fingered robotic hands to learn in‐hand manipulation effectively.