A V2G-oriented reinforcement learning framework and empirical study for heterogeneous electric vehicle charging management. (February 2023)

• Record Type: Journal Article
• Authors: Hao, Xu; Chen, Yue; Wang, Hewu; Wang, Han; Meng, Yu; Gu, Qing
• Is Part Of: Sustainable cities and society. Volume 89(2023)
• Publication Date: 2023-02
• Subjects: V2G (vehicle-to-grid) -- EV (electric vehicle) -- Charging -- Deep Q-network -- Travel pattern heterogeneity; Sustainable urban development -- Periodicals; Sustainable buildings -- Periodicals; Urban ecology (Sociology) -- Periodicals; 307.76
• DOI: 10.1016/j.scs.2022.104345
• Languages: English
• ISSNs: 2210-6707

Abstract

Abstract: Vehicle-to-grid (V2G) technology is a promising solution to energy supply security issues associated with future electric grids. A decisive factor to successful V2G is effective electric vehicle (EV) charging management aimed at meeting travel demands with minimal charging costs, especially how to account for uncertainties and EV heterogeneity. In this study, a deep Q-network (DQN)-based reinforcement learning (RL) method is proposed to learn the optimal EV charging strategy considering empirical travel pattern heterogeneities and unpredictable electricity prices. The effectiveness and generalizability of the proposed DQN-based RL method was validated using actual five-million-km driving data in typical Chinese cities. In particular, EVs can save over 98% of the electricity cost without future electricity price information via the proposed method compared to the charging as-soon-as-possible method. The empirical experimental results also reveal that V2G-oriented charging management is sensitive to the charging/discharging power rate, electricity-price fluctuation frequency and range, and departure-time. We quantified the sensitivity with value of information (VOI) and found that: (1) Knowing the departure-time information can significantly reduce charging costs in most cases (average VOI: 5.4 CNY per charging/discharging session); (2) More historical data does not always lead to a higher electricity price VOI, and prices with sudden surges may even have a negative VOI. Graphical abstract: Highlights: A learning-based EV charging method with travel pattern heterogeneity is developed. The method is validated for V2G with empirical travel data in typical cities. EVs save over 98% of electricity cost without future electricity price information. The values of departure time and electricity price information are quantified. Pre-decided departure-time can significantly reduce charging costs in most cases.