Erlang planning network: An iterative model-based reinforcement learning with multi-perspective. (August 2022)

• Record Type: Journal Article
• Authors: Wang, Jiao; Zhang, Lemin; He, Zhiqiang; Zhu, Can; Zhao, Zihui
• Is Part Of: Pattern recognition. Volume 128(2022)
• Publication Date: 2022-08
• Subjects: Model-based reinforcement learning -- Multi-perspective -- Bi-level -- Planning -- Trajectory imagination; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108668
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Multi-perspective fusion attenuates the impact of model error in model-based reinforcement learning. Bi-level architecture is used to integrate decisions under multiple views. Good generality for different kinds of robot control tasks. Dramatically improve performance without more sampling in the environment. Abstract: For model-based reinforcement learning (MBRL), one of the key challenges is modeling error, which cripples the effectiveness of model planning and causes poor robustness during training. In this paper, we propose a bi-level Erlang Planning Network (EPN) architecture, which is composed of an upper-level agent and several multi-scale parallel sub-agents, trained in an iterative way. The proposed method focuses upon the expansion of representation by environment: a multi-perspective over the world model, which presents a varied way to represent an agent's knowledge about the world that alleviates the problem of falling into local optimal points and enhances robustness during the progress of model planning. Moreover, our experiments evaluate EPN on a range of continuous-control tasks in MuJoCo, the evaluation results show that the proposed framework finds exemplar solutions faster and consistently reaches the state-of-the-art performance.