Learning manipulation skills from a single demonstration. (January 2018)

• Record Type: Journal Article
• Authors: Englert, Peter; Toussaint, Marc
• Is Part Of: International journal of robotics research. Volume 37:Number 1(2018)
• Page Start: 137
• Page End: 154
• Publication Date: 2018-01
• Subjects: Combined optimization and learning -- reinforcement learning -- imitation learning -- manipulation skills; Robots -- Periodicals; Robots, Industrial -- Periodicals; 629.89205
• DOI: 10.1177/0278364917743795
• Languages: English
• ISSNs: 0278-3649

Abstract

We consider the scenario where a robot is demonstrated a manipulation skill once and should then use only a few trials on its own to learn to reproduce, optimize, and generalize that same skill. A manipulation skill is generally a high-dimensional policy. To achieve the desired sample efficiency, we need to exploit the inherent structure in this problem. With our approach, we propose to decompose the problem into analytically known objectives, such as motion smoothness, and black-box objectives, such as trial success or reward, depending on the interaction with the environment. The decomposition allows us to leverage and combine (i) constrained optimization methods to address analytic objectives, (ii) constrained Bayesian optimization to explore black-box objectives, and (iii) inverse optimal control methods to eventually extract a generalizable skill representation. The algorithm is evaluated on a synthetic benchmark experiment and compared with state-of-the-art learning methods. We also demonstrate the performance on real-robot experiments with a PR2.