Deep reinforcement learning-based joint load scheduling for household multi-energy system. (15th October 2022)

• Record Type: Journal Article
• Authors: Zhao, Liyuan; Yang, Ting; Li, Wei; Zomaya, Albert Y.
• Is Part Of: Applied energy. Volume 324(2022)
• Publication Date: 2022-10-15
• Subjects: AC air conditioner -- CREST Center for Renewable Energy Systems Technology -- DDPG deep deterministic policy gradient -- DQN deep Q network -- DRL deep reinforcement learning -- EB electric boiler -- GAE generalized advantage estimation -- GS gas stove -- HMES household multi-energy system -- HVAC Heating, Ventilation, and Air Conditioning -- IC induction cooker -- IES integrated energy system -- MDP Markov decision process -- MPC model predictive control -- PPO proximal policy optimization -- PV photovoltaic -- RH resistance water heater -- RL reinforcement learning -- TRPO trust domain policy optimization -- WGB wall-hung gas boiler -- WM washing machine; Household multi-energy system -- Joint load scheduling -- Deep reinforcement learning -- Energy management; Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2022.119346
• Languages: English
• ISSNs: 0306-2619

Abstract

Abstract: Under the background of the popularization of renewable energy sources and gas-fired domestic devices in households, this paper proposes a joint load scheduling strategy for household multi-energy system (HMES) aiming at minimizing residents' energy cost while maintaining the thermal comfort. Specifically, the studied HMES contains photovoltaic, gas-electric hybrid heating system, gas-electric kitchen stove and various types of conventional loads. Yet, it is challenging to develop an efficient energy scheduling strategy due to the uncertainties in energy price, photovoltaic generation, outdoor temperature, and residents' hot water demand. To tackle this problem, we formulate the HMES scheduling problem as a Markov decision process with both continuous and discrete actions and propose a deep reinforcement learning-based HMES scheduling approach. A mixed distribution is used to approximate the scheduling strategies of different types of household devices, and proximal policy optimization is used to optimize the scheduling strategies without requiring any prediction information or distribution knowledge of system uncertainties. The proposed approach can handle continuous actions of power-shiftable devices and discrete actions of time-shiftable devices simultaneously, as well as the optimal management of electrical devices and gas-fired devices, so as to jointly optimize the operation of all household loads. The proposed approach is compared with a deep Q network (DQN)-based approach and a model predictive control (MPC)-based approach. Comparison results show that the average energy cost of the proposed approach is reduced by 12.17% compared to the DQN-based approach and 4.59% compared to the MPC-based approach.