Reinforcement learning for control of flexibility providers in a residential microgrid. Issue 1 (12th November 2019)

• Record Type: Journal Article
• Authors: Mbuwir, Brida V.; Geysen, Davy; Spiessens, Fred; Deconinck, Geert
• Is Part Of: IET smart grid. Volume 3:Issue 1(2020)
• Page Start: 98
• Page End: 107
• Publication Date: 2019-11-12
• Subjects: learning (artificial intelligence) -- multi‐agent systems -- power consumption -- distributed power generation -- photovoltaic power systems -- power engineering computing -- iterative methods -- power generation scheduling -- heat pumps -- stochastic processes -- control engineering computing -- power generation control; residential microgrid -- smart grid paradigm -- smart meters -- machine learning -- model‐free reinforcement learning techniques -- single‐agent stochastic microgrid settings -- rule‐based controller -- model‐based optimal controller -- electricity consumption patterns -- power system planning -- RL techniques -- policy iteration -- PI -- fitted Q‐iteration -- FQI -- heat pump -- multiagent collaborative microgrid settings -- photovoltaic production; B0240Z Other topics in statistics -- B0290F Interpolation and function approximation (numerical analysis) -- B8110C Power system control -- B8250 Solar power stations and photovoltaic power systems -- C1140Z Other topics in statistics -- C1340B Multivariable control systems -- C3340H Control of electric power systems -- C4130 Interpolation and function approximation (numerical analysis) -- C6170K Knowledge engineering techniques -- C7410B Power engineering computing -- C7420 Control engineering computing -- B8120K Distributed power generation; Smart power grids -- Periodicals; Computer science -- Periodicals; Energy industries -- Periodicals; Broadcasting -- Periodicals; 333.79110285
• DOI: 10.1049/iet-stg.2019.0196
• Languages: English
• ISSNs: 2515-2947

Abstract

Abstract : The smart grid paradigm and the development of smart meters have led to the availability of large volumes of data. This data is expected to assist in power system planning/operation and the transition from passive to active electricity users. With recent advances in machine learning, this data can be used to learn system dynamics. This study explores two model‐free reinforcement learning (RL) techniques – policy iteration (PI) and fitted Q‐iteration (FQI) for scheduling the operation of flexibility providers – battery and heat pump in a residential microgrid. The proposed algorithms are data‐driven and can be easily generalised to fit the control of any flexibility provider without requiring expert knowledge to build a detailed model of the flexibility provider and/or microgrid. The algorithms are tested in multi‐agent collaborative and single‐agent stochastic microgrid settings – with the uncertainty due to lack of knowledge on future electricity consumption patterns and photovoltaic production. Simulation results show that PI outperforms FQI with a 7.2% increase in photovoltaic self‐consumption in the multi‐agent setting and a 3.7% increase in the single‐agent setting. Both RL algorithms perform better than a rule‐based controller, and compete with a model‐based optimal controller, and are thus, a valuable alternative to model‐ and rule‐based controllers.