Deep imitation reinforcement learning with expert demonstration data. Issue 16 (31st October 2018)

• Record Type: Journal Article
• Authors: Yi, Menglong; Xu, Xin; Zeng, Yujun; Jung, Seul
• Is Part Of: Journal of engineering. Volume 2018:Issue 16(2018)
• Page Start: 1567
• Page End: 1573
• Publication Date: 2018-10-31
• Subjects: learning (artificial intelligence) -- computer games; expert demonstration data -- existing DRL algorithms -- good action policy -- task requirements -- expert data -- deep imitation reinforcement learning algorithm -- learning agent -- DIRL algorithm -- expert guidance -- Mario racing game; Engineering -- Periodicals; Engineering; Electronic journals; Periodicals; 620.005
• DOI: 10.1049/joe.2018.8314
• Languages: English
• ISSNs: 2051-3305

Abstract

Abstract : In recent years, deep reinforcement learning (DRL) has made impressive achievements in many fields. However, existing DRL algorithms usually require a large amount of exploration to obtain a good action policy. In addition, in many complex situations, the reward function cannot be well designed to meet task requirements. These two problems will make it difficult for DRL to learn a good action policy within a relatively short period. The use of expert data can provide effective guidance and avoid unnecessary exploration. This study proposes a deep imitation reinforcement learning (DIRL) algorithm that uses a certain amount of expert demonstration data to speed up the training of DRL. In the proposed method, the learning agent imitates the expert's action policy by learning from demonstration data. After imitation learning, DRL is used to optimise the action policy in a self‐learning way. By experimental comparison on a video game called the Mario racing game, it is shown that the proposed DIRL algorithm with expert demonstration data can obtain much better performance than previous DRL algorithms without expert guidance.