A reinforcement learning approach to home energy management for modulating heat pumps and photovoltaic systems. (1st December 2022)

• Record Type: Journal Article
• Authors: Langer, Lissy; Volling, Thomas
• Is Part Of: Applied energy. Volume 327(2022)
• Publication Date: 2022-12-01
• Subjects: Home energy management -- Building energy management -- Heat pump -- Photovoltaics (PV) -- Reinforcement learning -- Deep deterministic policy gradient (DDPG); Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2022.120020
• Languages: English
• ISSNs: 0306-2619

Abstract

Abstract: Buildings are one of the main drivers of global energy consumption and CO 2 emissions. Efficient energy management systems will have to integrate renewable energy sources with heating and/or cooling to mitigate climate change. In this study, we analyze the potential of deep reinforcement learning (DRL) to control a smart home with a modulating air-to-water heat pump, a photovoltaic system, a battery energy, and a thermal storage system for floor heating and hot water supply. We transform a mixed-integer linear program (MILP) into a DRL implementation. In our numerical analysis, we compare our results based on the deep deterministic policy gradient (DDPG) algorithm to the theoretical upper bound of the model predictive control (MPC) result under full information, as well as a practice-oriented rule-based benchmark. We show that our proposed DRL implementation outperforms the rule-based approach and achieves a self-sufficiency of 75% with only limited comfort violations. Analyzing different DRL formulations, we conclude that domain knowledge is key to formalizing an efficient decision problem with stable results. Our input data and models, developed using the Julia programming language, are available open source. Highlights: RL environment proposed for an integrated home energy management system. Structured solution approach using DDPG algorithm for stable results. RL's dynamic target states-of-charge outperform practice-oriented rules. Data, as well as Julia code of RL model and environment provided open source.