Dynamic energy scheduling and routing of a large fleet of electric vehicles using multi-agent reinforcement learning. (July 2022)

• Record Type: Journal Article
• Authors: Alqahtani, Mohammed; Scott, Michael J.; Hu, Mengqi
• Is Part Of: Computers & industrial engineering. Volume 169(2022)
• Publication Date: 2022-07
• Subjects: Electric vehicle -- Vehicle routing -- Energy scheduling -- Multi-agent reinforcement learning -- Deep reinforcement learning; Engineering -- Data processing -- Periodicals; Industrial engineering -- Periodicals; 620.00285
• DOI: 10.1016/j.cie.2022.108180
• Languages: English
• ISSNs: 0360-8352

Abstract

Highlights: An vehicle routing and energy scheduling decision model is developed. The proposed model enables a flexable energy sharing among a large number of EV's. The proposed algorithm outperforms heuristic algorithms in terms of solution quality. The proposed model is more computational efficient than heuristic algorithms and CRL. The proposed algorithm output less solution quality compared to CRL. Abstract: As the world's population and economy grow, demand for energy increases as well. Smart grids can be a cost-effective solution to overcome increases in energy demand and ensure power security. Current applications of smart grids involve a large numbers of agents (e.g., electric vehicles). Since each agent must interact with other agents when taking decisions (e.g., movement and scheduling), the computational complexity of smart grid systems increases exponentially with the number of agents. Computational tractability of planning is a significant barrier to implementation of large-scale smart grids of electric vehicles. Existing solution approaches such as mixed-integer programming and dynamic programming are not computationally efficient for high-dimensional problems. This paper proposes a reformulation of a Mixed-Integer Programming model into a Decentralized Markov Decision Process model and solves it using a Multi-Agent Reinforcement Learning algorithm to address the scalability issues of large-scale smart grid systems. The Decentralized Markov Decision Process model uses centralized training and distributed execution: agents are trained using a unique actor network for each agent and a shared critic network, and then agent execute actions independently from other agents to reduce computation time. The performance of the Multi-Agent Reinforcement Learning model is assessed under different configurations of customers and electric vehicles, and compared to the results from deep reinforcement learning and three heuristic algorithms. The simulation results demonstrate that the Multi-Agent Reinforcement Learning algorithm can reduce simulation time significantly compared to deep reinforcement learning, genetic algorithm, particle swarm optimization, and the artificial fish swarm algorithm. The superior performance of the proposed method indicates that it may be a realistic solution for large-scale implementation.