Towards healthy and cost-effective indoor environment management in smart homes: A deep reinforcement learning approach. (15th October 2021)

• Record Type: Journal Article
• Authors: Yang, Ting; Zhao, Liyuan; Li, Wei; Wu, Jianzhong; Zomaya, Albert Y.
• Is Part Of: Applied energy. Volume 300(2021)
• Publication Date: 2021-10-15
• Subjects: Indoor environment management -- Deep reinforcement learning -- Healthy -- Energy cost -- Optimal control -- Smart home; Power (Mechanics) -- Periodicals; Energy conservation -- Periodicals; Energy conversion -- Periodicals; 621.042
• DOI: 10.1016/j.apenergy.2021.117335
• Languages: English
• ISSNs: 0306-2619

Abstract

Highlights: A deep reinforcement learning strategy for indoor environment management is proposed. Indoor air quality and thermal comfort are maintained with minimized energy cost. Uncertainties of weather, electricity price and home occupancy are considered. The results validate the adaptability of the method to the variation of variables. Average energy cost reduces 8.56% and 7.88% than MPC method in winter and summer. Abstract: Indoor environmental quality is an important issue since people spend most of their time indoors. This paper aims to develop an autonomous indoor environment management approach to ensure a healthy indoor environment with minimized energy cost via the optimal control of ventilation system and heating/cooling system in smart homes. Due to the uncertainties of weather conditions, electricity price and home occupancy, as well as the complex interaction between indoor air quality and indoor thermal environment, it is challenging to develop an efficient control strategy. To address this challenge, the indoor environment management problem is formulated as a Markov decision process, and then a deep reinforcement learning control strategy, which combines double deep Q network with prioritized experience replay mechanism, is proposed to solve the Markov decision process. The proposed approach can make adaptive control decisions based on the current observations without requiring any forecast information of system uncertainties. Control performance under different scenarios show the proposed approach has good adaptability to the variation of weather conditions, electricity prices, home occupancy patterns and indoor temperature requirements. Moreover, the proposed approach is compared with a double deep Q network-based approach and a model predictive control-based approach. Comparison results show that the proposed approach reduces the average daily energy cost by 3.51% and 8.56% in winter scenarios and 4.05%, 7.88% in summer scenarios while achieving the smallest indoor air quality and temperature violations.