An integrated critic-actor neural network for reinforcement learning with application of DERs control in grid frequency regulation. (October 2019)

• Record Type: Journal Article
• Authors: Sun, Jian; Zhu, Zhiqin; Li, Huaqing; Chai, Yi; Qi, Guanqiu; Wang, Huiwei; Hu, Yu Hen
• Is Part Of: International journal of electrical power & energy systems. Volume 111(2019)
• Page Start: 286
• Page End: 299
• Publication Date: 2019-10
• Subjects: Power grid -- Control theory -- Reinforcement learning -- Neural networks; Electrical engineering -- Periodicals; Electric power systems -- Periodicals; Électrotechnique -- Périodiques; Réseaux électriques (Énergie) -- Périodiques; Electric power systems; Electrical engineering; Periodicals; 621.3
• DOI: 10.1016/j.ijepes.2019.04.011
• Languages: English
• ISSNs: 0142-0615

Abstract

Highlights: An integrated actor-critic neural network is proposed for power grid frequency regulation by DERs. An on-line reinforcement learning scheme is presented for improving performance of controller. Stability conditions of proposed scheme are derived and the control performance is estimated. Abstract: As the electronically-interfaced distributed energy resources (DERs) grow rapidly in power grid, power demand satisfaction and frequency regulation are two main challenges in control area. However, it is difficult to model the analysis of a large-scale grid as well as design a stable and optimal control scheme. With the support of DERs, this paper proposes an actor-critic neural network that integrates a distributed reinforcement learning control scheme to compensate frequency regulation of power grid. The short-term performance and stability is improved by a deterministic learning algorithm that is used to obtain the approximation of desired control output. Meanwhile, a long-term strategic utility function is estimated by the integrated actor-critic neural network. The mapping from system state and control output to the strategic utility function value is identified by neural network, as well as utilized in sub-optimal control learning for further improvement of long-term system performance. Theoretical analysis guarantees the stability. Frequency deviation, tie-line power flow, and long-term cost are coincident with uniform ultimate boundness (UUB). In addition, the upper bound of long-term system cost is also reckoned. The effectiveness and advantages of proposed scheme are illustrated in two case studies. The simulation results indicate that the proposed scheme has better performance under certain condition, compared with some actor-critic network control schemes in frequency regulation of power grid.