A bi-level reinforcement learning model for optimal scheduling and planning of battery energy storage considering uncertainty in the energy-sharing community. (July 2023)

• Record Type: Journal Article
• Authors: Kang, Hyuna; Jung, Seunghoon; Jeoung, Jaewon; Hong, Juwon; Hong, Taehoon
• Is Part Of: Sustainable cities and society. Volume 94(2023)
• Publication Date: 2023-07
• Subjects: Battery energy storage system -- Reused battery -- Optimal scheduling -- Optimal planning -- Reinforcement learning -- Geographic information system; BESS Battery energy storage system -- BESSNew Battery energy storage system using new batteries -- BESSReused Battery energy storage system using reused batteries -- CAPEX Capital expenditures -- DEM Digital evaluation model -- DSM Digital surface model -- EVs Electric vehicles -- GAE Generalized advantage estimation -- GBM Geometric brownian motion -- GIS Geographic information system -- IEA International energy agency -- IPCC Intergovernmental panel on climate change -- KEPCO Korea electric power corporation -- LCC Life cycle cost -- LiDAR Light detection and ranging -- OPEX Operational expenditure -- PPO Proximal policy optimization -- PV Photovoltaic -- RCP Representative concentration pathways -- REC Renewable energy certificates -- RL Reinforcement learning -- SMpeak Scheduling model for minimizing peak demand -- SMprofit Scheduling model for maximizing economic profit -- SMSSR Scheduling model for maximizing self-sufficiency rate -- SMP System marginal price -- SoC State of charge -- SoH State of health -- SSR Self-sufficiency rate -- PM-SMpeak Planning model applying scheduling model for minimizing peak demand -- PM-SMprofit Planning model applying scheduling model for maximizing economic profit -- PM-SMSSR Planning model applying scheduling model for maximizing self-sufficiency rate; Sustainable urban development -- Periodicals; Sustainable buildings -- Periodicals; Urban ecology (Sociology) -- Periodicals; 307.76
• DOI: 10.1016/j.scs.2023.104538
• Languages: English
• ISSNs: 2210-6707

Abstract

Highlights: A bi-level reinforcement learning model for battery energy storage was developed. Short-term scheduling model optimizes electric flow considering operation objectives. Long-term planning model determines the annual battery plan along with battery types. Case study was conducted based on future scenarios for community-shared battery. This study can increase economic profit by up to 18, 830 USD by reflecting real world. Abstract: Sharing of battery energy storage systems (BESS) in the energy community by reflecting the real world can play a significant role in achieving carbon neutrality. Therefore, this study aimed to develop a bi-level reinforcement learning (RL) model of BESS considering uncertainty in the energy-sharing community for the following optimization strategies: (i) short-term scheduling model for optimal electricity flows considering operational objectives (i.e., self-sufficiency rate (SSR), peak load, and economic profit); and (ii) long-term planning model for optimal BESS plan (i.e., install, replace, and disuse) along with battery types (new or reused batteries). A case study in the South Korea Nonhyeon neighborhood was conducted to evaluate the developed bi-level RL model feasibility based on future scenarios considering the time-dependent variables. The developed model increased economic profit by up to 18, 830 USD compared to the rule-based model. Compared to the case where BESS was not installed, SSR increased by up to 7.79% and peak demand decreased by up to 1.31 kWh. These results show that the developed model could maximize the economic feasibility of community-shared BESS by reflecting the uncertainty in the real world, ultimately benefiting participants in the energy-sharing community.