Genetic state-grouping algorithm for deep reinforcement learning. (15th December 2020)

• Record Type: Journal Article
• Authors: Kim, Man-Je; Kim, Jun Suk; Kim, Sungjin James; Kim, Min-jung; Ahn, Chang Wook
• Is Part Of: Expert systems with applications. Volume 161(2020)
• Publication Date: 2020-12-15
• Subjects: Reinforcement learning -- Genetic algorithm -- Hybrid method -- Monte Carlo Tree Search -- Game AI; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2020.113695
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Genetic-State Grouping Algorithm guarantees enhancing the performance of RL agents. The genetic algorithm has successfully been combined with Monte Carlo Tree Search. Video game provides a valid proving ground for testing AI's capability. Abstract: Although Reinforcement learning has already been considered one of the most important and well-known techniques of machine learning, its applicability remains limited in the real-world problems due to its long initial learning time and unstable learning. Especially, the problem of an overwhelming number of the branching factors under real-time constraint still stays unconquered, demanding a new method for the next generation of reinforcement learning. In this paper, we propose Genetic State-Grouping Algorithm based on deep reinforcement learning. The core idea is to divide the entire set of states into a few state groups. Each group consists of states that are mutually similar, thus representing their common features. The state groups are then processed with the Genetic Optimizer, which finds outstanding actions. These steps help the Deep Q Network avoid excessive exploration, thereby contributing to the significant reduction of initial learning time. The experiment on the real-time fighting video game (FightingICE) shows the effectiveness of our proposed approach.