Application of advanced tree search and proximal policy optimization on formula-E race strategy development. (1st July 2022)

• Record Type: Journal Article
• Authors: Liu, Xuze; Fotouhi, Abbas; Auger, Daniel
• Is Part Of: Expert systems with applications. Volume 197(2022)
• Publication Date: 2022-07-01
• Subjects: Energy management -- Formula-E race strategy -- Monte Carlo Tree search -- Proximal policy optimization; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.116718
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Targeting the most popular strategic topic in Formula E championship races. Proposed two enhancement techniques to Monte Carlo Tree Search. MCTS performance remarkably improves using Bivariate Gaussian distribution. Integrating MCTS and PPO significantly improves race time solution and consistency. Abstract: Energy and thermal management is a crucial element in Formula-E race strategy development. Most published literature focuses on the optimal management strategy for a single lap and results in sub-optimal solutions to the larger multi-lap problem. In this study, two Monte Carlo Tree Search (MCTS) enhancement techniques are proposed for multi-lap Formula-E racing strategy development. It is shown that using the bivariate Gaussian distribution enhancement, race finishing time improves by at least 0.25% and its variance reduces by more than 26%. Compared to the published conventional MCTS technique used in multi-lap problems, this proposed technique is proved to bring a remarkable enhancement with no additional computational time cost. By further enhancing the MCTS using proximal policy optimization, the final product is capable of generating more than 0.5% quicker race time solutions and improving the consistency by over 90% which makes it a very suitable method particularly when enough training time is guaranteed