Data-driven stochastic energy management of multi energy system using deep reinforcement learning. (15th December 2022)

• Record Type: Journal Article
• Authors: Zhou, Yanting; Ma, Zhongjing; Zhang, Jinhui; Zou, Suli
• Is Part Of: Energy. Volume 261:Part B(2022)
• Publication Date: 2022-12-15
• Subjects: Carbon neutrality -- Multi energy system -- Renewable energy -- Stochastic optimization -- Deep reinforcement learning -- Soft actor critic; Power resources -- Periodicals; Power (Mechanics) -- Periodicals; Energy consumption -- Periodicals; 333.7905
• DOI: 10.1016/j.energy.2022.125187
• Languages: English
• ISSNs: 0360-5442

Abstract

Abstract: The multi energy system (MES) is promising in the process of carbon neutrality, such that multi energy resources are utilized comprehensively to reduce the operation cost. Another way is to promote carbon neutrality by increasing the penetration of renewable energy. Hence, in this paper, we study the energy management of a typical MES under the challenges of stochastic renewable supplies and energy demands. To address the challenges, a stochastic optimization problem is established as a Markov decision process (MDP). An improved deep reinforcement learning (DRL) method is then developed to achieve the dynamic optimal energy dispatch. In particular, the comfort experience of users and complex coupling are both considered in the MES. In this framework, we propose an improved soft actor critic (SAC) algorithm based on maximum entropy to improve exploration ability, together with a long short-term memory (LSTM) network to extract temporal features efficiently. Meanwhile, we add the prioritized experience replay (PER) to increase the training efficiency to speed up the convergence of the algorithm. Finally, the case study demonstrates that the proposed algorithm can converge rapidly and greatly reduce the operation cost. In addition, the effectiveness and robustness of the improved method are verified. Highlights: A stochastic optimization model is mapped to a Markov decision process. Uncertainties of renewable supplies and energy demands are considered. An improved deep reinforcement learning algorithm based on PLSAC is applied. The effectiveness of the optimal dispatch in the multi energy system is validated.