Data-driven online interactive bidding strategy for demand response. (1st August 2022)

• Record Type: Journal Article
• Authors: Lee, Kuan-Cheng; Yang, Hong-Tzer; Tang, Wenjun
• Is Part Of: Applied energy. Volume 319(2022)
• Publication Date: 2022-08-01
• Subjects: Smart meter -- Demand response -- Demand bidding -- Machine learning -- Multi-agent Reinforcement Learning; Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2022.119082
• Languages: English
• ISSNs: 0306-2619

Abstract

Abstract: Demand response (DR), as one of the important energy resources in the future's grid, provides the services of peak shaving, enhancing the efficiency of renewable energy utilization with a short response period, and low cost. Various categories of DR are established, e.g. automated DR, incentive DR, emergency DR, and demand bidding. However, with the practical issue of the unawareness of residential and commercial consumers' utility models, the researches about demand bidding aggregator involved in the electricity market are just at the beginning stage. For this issue, the bidding price and bidding quantity are two required decision variables while considering the uncertainties due to the market and participants. In this paper, we determine the bidding and purchasing strategy simultaneously employing the smart meter data and functions. A two-agent deep deterministic policy gradient method is developed to optimize the decisions through learning historical bidding experiences. The online learning further utilizes the daily newest bidding experience attained to ensure trend tracing and self-adaptation. Two environment simulators are adopted for testifying the robustness of the model. The results prove that when facing diverse situations the proposed model can earn the optimal profit via off/online learning the bidding rules and robustly making the proper bid. Graphical abstract: Highlights: The proposed Reinforcement Learning-based model generates the bidding strategy for demand response aggregator, which achieves the win–win solution for both consumers and aggregator. The two-agent structure of the model considers the uncertainties introduced from the market and consumer. The self-adjustment and learning ability are both characterized by the structure. The strategy is proved to be robust with the online learning ability while the simulation results exhibit the adaptivity and the profit-pursuing ability of the model facing various uncertainties.