Teaching a humanoid robot to walk faster through Safe Reinforcement Learning. (February 2020)

• Record Type: Journal Article
• Authors: García, Javier; Shafie, Diogo
• Is Part Of: Engineering applications of artificial intelligence. Volume 88(2020)
• Publication Date: 2020-02
• Subjects: Safe Reinforcement Learning -- Biped walking; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2019.103360
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: Teaching a humanoid robot to walk is an open and challenging problem. Classical walking behaviors usually require the tuning of many control parameters (e.g., step size, speed). To find an initial or basic configuration of such parameters could not be so hard, but optimizing them for some goal (for instance, to walk faster) is not easy because, when defined incorrectly, may produce the fall of the humanoid, and the consequent damages. In this paper we propose the use of Safe Reinforcement Learning for improving the walking behavior of a humanoid that permits the robot to walk faster than with a pre-defined configuration. Safe Reinforcement Learning assumes the existence of a safe baseline policy that permits the humanoid to walk, and probabilistically reuse such a policy to learn a better one, which is represented following a case based approach. The proposed algorithm has been evaluated in a real humanoid robot proving that it drastically increases the learning speed while reduces the number of falls during learning when compared with state-of-the-art algorithms. Highlights: Reinforcement Learning is used to solve the biped robot walking problem. The algorithm PR-SRL is successfully used to solve the problem. PR-SRL performs a safe exploration of the state and action spaces. PR-SRL maximizes the traveled distance and avoid the falls during learning. We use a real NAO robot to perform the experiments.