Constrained large-scale real-time EV scheduling based on recurrent deep reinforcement learning. (January 2023)

• Record Type: Journal Article
• Authors: Li, Hang; Li, Guojie; Lie, Tek Tjing; Li, Xingzhi; Wang, Keyou; Han, Bei; Xu, Jin
• Is Part Of: International journal of electrical power & energy systems. Volume 144(2023)
• Publication Date: 2023-01
• Subjects: Electric vehicles (EVs) charging scheduling -- Recurrent deep deterministic policy gradient (RDDPG) -- Markov decision process (MDP) -- Large-scale; Electrical engineering -- Periodicals; Electric power systems -- Periodicals; Électrotechnique -- Périodiques; Réseaux électriques (Énergie) -- Périodiques; Electric power systems; Electrical engineering; Periodicals; 621.3
• DOI: 10.1016/j.ijepes.2022.108603
• Languages: English
• ISSNs: 0142-0615

Abstract

Highlights: A decentralized framework for large-scale EV scheduling. A DEB model discretizing use's charging demand and the charging capacity of the CS. RDDPG method to train one agent model to schedule multi charging piles. Significant improvement in scalability for large-scale EVs scenario. Abstract: The rapid growth of electric vehicles (EVs) is an unstoppable worldwide development trend. An optimal charging strategy for large-scale EVs is able to deal with the randomness of EVs charging and satisfy charging demands of users while ensuring safe and economic operation of the power system. The current centralized and model-based methods failed to overcome the randomness charging problem of the large-scale EVs. Thus, the paper proposes a decentralized approach based on model-free deep reinforcement learning (DRL) to determine the optimal strategy for reducing EVs charging cost considering power limit of the charging station (CS), users' charging demands and fair charging fees. First, a decentralized framework and a dynamic energy boundary (DEB) model of single EV which discretizes the charging demand are proposed. Second, the problem as a Markov Decision Process (MDP) with unknown transition probability is formulated. Moreover, the recurrent deep deterministic policy gradient (RDDPG) based approach is proposed to determine the charging strategy for all charging piles in the CS. Finally, digital simulation studies are conducted to demonstrate the effectiveness of the proposed approach in charging cost reduction and fair charging fees. In addition, the RDDPG-based approach has great scalability which can apply a small-scale model to solve a large-scale problem without being retrained.