State-space segmentation for faster training reinforcement learning. Issue 25 (2022)

• Record Type: Journal Article
• Authors: Kim, Jongrae
• Is Part Of: IFAC-PapersOnLine. Volume 55:Issue 25(2022)
• Page Start: 235
• Page End: 240
• Publication Date: 2022
• Subjects: reinforcement learning -- learning convergence -- reward -- linear control; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2022.09.352
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: Nonlinear control problems have been the main subjects in control engineering from theoretical and applicational aspects. Reinforcement learning shows promising results for solving highly nonlinear control problems. Among many variants of reinforcement learning, Deep Deterministic Policy Gradient (DDPG) considers continuous control signals, which makes it an ideal candidate for solving nonlinear control problems. The training requires frequently, however, a large number of computations. To improve the convergence of DDPG, we present a state-space segmentation method dividing the state-space to expand the target space defined by the best reward. An inverted pendulum control example demonstrates the performance of the proposed segmentation method.