A novel reinforcement learning method for improving occupant comfort via window opening and closing. (October 2020)

• Record Type: Journal Article
• Authors: Han, Mengjie; May, Ross; Zhang, Xingxing; Wang, Xinru; Pan, Song; Da, Yan; Jin, Yuan
• Is Part Of: Sustainable cities and society. Volume 61(2020)
• Publication Date: 2020-10
• Subjects: Markov decision processes -- Reinforcement learning -- Window control -- Indoor comfort -- Occupant; Sustainable urban development -- Periodicals; Sustainable buildings -- Periodicals; Urban ecology (Sociology) -- Periodicals; 307.76
• DOI: 10.1016/j.scs.2020.102247
• Languages: English
• ISSNs: 2210-6707

Abstract

Highlights: The model-free RL method is developed for controlling windows to improve the indoor comfort. Representative factors of thermal comfort and indoor air quality are considered for training agents and evaluating policies. LSTM RNN is able to predict indoor temperature and simulate the building environment. The feasibility of Q-learning and SARSA algorithms is tested. Abstract: An occupant's window opening and closing behaviour can significantly influence the level of comfort in the indoor environment. Such behaviour is, however, complex to predict and control conventionally. This paper, therefore, proposes a novel reinforcement learning (RL) method for the advanced control of window opening and closing. The RL control aims at optimising the time point for window opening/closing through observing and learning from the environment. The theory of model-free RL control is developed with the objective of improving occupant comfort, which is applied to historical field measurement data taken from an office building in Beijing. Preliminary testing of RL control is conducted by evaluating the control method's actions. The results show that the RL control strategy improves thermal and indoor air quality by more than 90% when compared with the actual historically observed occupant data. This methodology establishes a prototype for optimally controlling window opening and closing behaviour. It can be further extended by including more environmental parameters and more objectives such as energy consumption. The model-free characteristic of RL avoids the disadvantage of implementing inaccurate or complex models for the environment, thereby enabling a great potential in the application of intelligent control for buildings.