Machine learning analytics for virtual bidding in the electricity market. (December 2022)

• Record Type: Journal Article
• Authors: Han, Dong; Huang, Wei; Ren, Hengyu; Zhao, Wenkai; Li, Yiyan
• Is Part Of: International journal of electrical power & energy systems. Volume 143(2022)
• Publication Date: 2022-12
• Subjects: Virtual bidding -- Markov decision process -- Deep reinforcement learning -- Conditional value at risk -- Sharpe ratio; Electrical engineering -- Periodicals; Electric power systems -- Periodicals; Électrotechnique -- Périodiques; Réseaux électriques (Énergie) -- Périodiques; Electric power systems; Electrical engineering; Periodicals; 621.3
• DOI: 10.1016/j.ijepes.2022.108489
• Languages: English
• ISSNs: 0142-0615

Abstract

Highlights: A framework of virtual bidding problem is established and the model is formulated as a MDP problem. A machine learning method is proposed to solve the virtual bidding problem from the spatio-temporal dimensions. The DQN algorithm improves the cumulative profits of virtual bidders and hedges the transaction risks effectively. The simulations are performed to verify the proposed method. Abstract: In order to solve the problem of the high risks and low efficiency caused by the inconsistency of the day-ahead and real-time prices in two-settlement electricity market, virtual bidding is used to arbitrage on the difference between such two market prices that are unknown to virtual bidders to promote the price convergence. The problem of optimal bidding for virtual bidders from the spatio-temporal dimensions is addressed in this paper. The model takes the budget constraints of virtual bidders into account, as well as considers decrement and increment bids of virtual bidding to maximize the cumulative payoff of virtual bidders, which is formulated as a Markov Decision Process problem. Meanwhile, the conditional value-at-risk is used to quantify and hedge the risks faced by virtual bidders. A deep reinforcement learning algorithm is used to achieve an effective solution to the optimal bidding strategy problem through continuous interaction with a simulated building environment to obtain feedback and update the parameters of the neural network without referring to any prior model knowledge. The PJM data from 2016 to 2018 is used to calculate the cumulative profits and Sharpe ratio of virtual bidders. Compared with greedy algorithm and dynamic programming, the deep reinforcement learning algorithm is verified the effectiveness and superiority in this paper.