Model-based reinforcement learning with dimension reduction. (December 2016)

• Record Type: Journal Article
• Authors: Tangkaratt, Voot; Morimoto, Jun; Sugiyama, Masashi
• Is Part Of: Neural networks. Volume 84(2016)
• Page Start: 1
• Page End: 16
• Publication Date: 2016-12
• Subjects: Model-based reinforcement learning -- Transition model estimation -- Sufficient dimension reduction; Neural computers -- Periodicals; Neural networks (Computer science) -- Periodicals; Neural networks (Neurobiology) -- Periodicals; Nervous System -- Periodicals; Ordinateurs neuronaux -- Périodiques; Réseaux neuronaux (Informatique) -- Périodiques; Réseaux neuronaux (Neurobiologie) -- Périodiques; Neural computers; Neural networks (Computer science); Neural networks (Neurobiology); Periodicals; 006.32
• DOI: 10.1016/j.neunet.2016.08.005
• Languages: English
• ISSNs: 0893-6080

Abstract

Abstract: The goal of reinforcement learning is to learn an optimal policy which controls an agent to acquire the maximum cumulative reward. The model-based reinforcement learning approach learns a transition model of the environment from data, and then derives the optimal policy using the transition model. However, learning an accurate transition model in high-dimensional environments requires a large amount of data which is difficult to obtain. To overcome this difficulty, in this paper, we propose to combine model-based reinforcement learning with the recently developed least-squares conditional entropy (LSCE) method, which simultaneously performs transition model estimation and dimension reduction. We also further extend the proposed method to imitation learning scenarios. The experimental results show that policy search combined with LSCE performs well for high-dimensional control tasks including real humanoid robot control.