Markov game approach for multi‐agent competitive bidding strategies in electricity market. Issue 15 (1st November 2016)

• Record Type: Journal Article
• Authors: Rashedi, Navid; Tajeddini, Mohammad Amin; Kebriaei, Hamed
• Is Part Of: IET generation, transmission & distribution. Volume 10:Issue 15(2016)
• Page Start: 3756
• Page End: 3763
• Publication Date: 2016-11-01
• Subjects: power markets -- power system economics -- power system analysis computing -- Markov processes -- game theory -- multi‐agent systems -- learning (artificial intelligence) -- decision making -- optimisation; Markov game model -- multiagent competitive bidding strategies -- competitive electricity market -- optimal bidding strategy -- individual profit maximisation -- straightforward optimisation methods -- supply function equilibrium model -- uniform price market clearing mechanism -- incomplete information game problem -- multiagent reinforcement learning -- noncooperative game -- single agent Markov decision process -- multiagent decision making problems -- learning‐based noncooperative MARL method -- IEEE‐30‐bus power system -- T‐test method; Electric power production -- Periodicals; Electric power transmission -- Periodicals; Electric power distribution -- Periodicals; 621.3105
• DOI: 10.1049/iet-gtd.2016.0075
• Languages: English
• ISSNs: 1751-8687

Abstract

Abstract : In a competitive electricity market, suppliers seek the optimal bidding strategy, maximising their individual profit. Due to the uncertainties, unknown parameters and dynamics of electricity market, the optimal bidding strategies cannot be attained through straightforward optimisation methods. In this study, the supply function equilibrium model is considered together with uniform price market clearing mechanism. The interaction among suppliers is modelled as an incomplete information game problem and multi‐agent reinforcement learning (MARL) is utilised to find the optimal bidding strategy of the suppliers in a non‐cooperative game. The proposed framework is an extension, from single agent Markov decision process to Markov game process, which is suitable for studying multi‐agent decision making problems in stochastic environments. In this study, a Markov game model of the electricity market is proposed and a new state‐action and Markov model of electricity market is proposed to built up the MARL environment. In addition, a learning‐based non‐cooperative MARL method is utilised for learning the optimal bidding strategies of the suppliers in a day‐ahead electricity market. The proposed method is successfully applied to IEEE‐30‐bus power system. To examine the statistical significance of the results, the T‐test method is applied to compare the competitive behaviour of the players in both multi‐ and single‐agent frameworks.