A reinforcement learning approach to coordinate exploration with limited communication in continuous action games. (11th February 2016)

• Record Type: Journal Article
• Authors: Rodríguez, Abdel; Vrancx, Peter; Grau, Ricardo; Nowé, Ann
• Is Part Of: Knowledge engineering review. Volume 31:Number 1(2016:Mar.)
• Page Start: 77
• Page End: 95
• Publication Date: 2016-02-11
• Subjects: Expert systems (Computer science) -- Periodicals; 006.33
• DOI: 10.1017/S026988891500020X
• Languages: English
• ISSNs: 0269-8889

Abstract

Abstract: Learning automata are reinforcement learners belonging to the class of policy iterators. They have already been shown to exhibit nice convergence properties in a wide range of discrete action game settings. Recently, a new formulation for a continuous action reinforcement learning automata (CARLA) was proposed. In this paper, we study the behavior of these CARLA in continuous action games and propose a novel method for coordinated exploration of the joint-action space. Our method allows a team of independent learners, using CARLA, to find the optimal joint action in common interest settings. We first show that independent agents using CARLA will converge to a local optimum of the continuous action game. We then introduce a method for coordinated exploration which allows the team of agents to find the global optimum of the game. We validate our approach in a number of experiments.