A deep reinforcement learning method for managing wind farm uncertainties through energy storage system control and external reserve purchasing. (July 2020)

• Record Type: Journal Article
• Authors: Yang, J.J.; Yang, M.; Wang, M.X.; Du, P.J.; Yu, Y.X.
• Is Part Of: International journal of electrical power & energy systems. Volume 119(2020)
• Publication Date: 2020-07
• Subjects: Deep reinforcement learning -- Energy storage system -- Optimal controller -- Rainbow -- Reserve -- Wind power producer; Electrical engineering -- Periodicals; Electric power systems -- Periodicals; Électrotechnique -- Périodiques; Réseaux électriques (Énergie) -- Périodiques; Electric power systems; Electrical engineering; Periodicals; 621.3
• DOI: 10.1016/j.ijepes.2020.105928
• Languages: English
• ISSNs: 0142-0615

Abstract

Highlights: A deep reinforcement learning method for wind farm revenues is proposed. The influence of the uncertainties can be involved without any assumption. Energy storage system control and reserve purchase are considered as control actions. Rainbow algorithm is implemented to improve the effectiveness of the method. Abstract: In deregulated environment, the wind power producers (WPPs) will face the challenge of how to increase their revenues under uncertainties of wind generation and electricity price. This paper proposes a method based on deep reinforcement learning (DRL) to address this issue. A data-driven controller that directly maps the input observations, i.e., the forecasted wind generation and electricity price, to the control actions of the wind farm, i.e., the charge/discharge schedule of the relevant energy storage system (ESS) and the reserve purchase schedule, is trained according to the method. By the well-trained controller, the influence of the uncertainties of wind power and electricity price on the revenue can be automatically involved and an expected optimal decision can be obtained. Furthermore, a targeted DRL algorithm, i.e., the Rainbow algorithm, is implemented to improve the effectiveness of the controller. Especially, the algorithm can overcome the limitation of the conventional reinforcement learning algorithms that the input states must be discrete, and thus the validity of the control strategy can be significantly improved. Simulation results illustrate that the proposed method can effectively cope with the uncertainties and bring high revenues to the WPPs.