A review of reinforcement learning methodologies for controlling occupant comfort in buildings. (November 2019)

• Record Type: Journal Article
• Authors: Han, Mengjie; May, Ross; Zhang, Xingxing; Wang, Xinru; Pan, Song; Yan, Da; Jin, Yuan; Xu, Liguo
• Is Part Of: Sustainable cities and society. Volume 51(2020)
• Publication Date: 2019-11
• Subjects: Reinforcement learning -- Control -- Building -- Indoor comfort -- Occupant; Sustainable urban development -- Periodicals; Sustainable buildings -- Periodicals; Urban ecology (Sociology) -- Periodicals; 307.76
• DOI: 10.1016/j.scs.2019.101748
• Languages: English
• ISSNs: 2210-6707

Abstract

Highlights: Empirical applications of RL-based control systems are presented depending on comfort objectives. The class of RL algorithms and implementation details are illustrated. Comfort objectives include thermal comfort, indoor air quality, and lighting. Relatively less works for RL are explored for controlling occupant comfort in IAQ and lighting. Abstract: Classical building control systems are becoming vulnerable with increasing complexities in contemporary built environments and energy systems. Due to this, the reinforcement learning (RL) method is becoming more distinctive and applicable in control networks for buildings. This paper, therefore, conducts a comprehensive review of RL techniques applied in control systems for occupant comfort in indoor built environments. The empirical applications of RL-based control systems are presented, depending on comfort objectives (thermal comfort, indoor air quality, and lighting) along with other objectives which invariably includes energy consumption. The class of RL algorithms and implementation details regarding how the value functions have been represented and how the policies are improved are also illustrated. This paper shows there are limited works for which RL has been explored for controlling occupant comfort, especially in indoor air quality and lighting. Relatively few of the reviewed works incorporate occupancy patterns and/or occupant feedback into the control loop. Moreover, this paper identifies a gap with regard to the performance of implementing cooperative multi-agent RL (MARL). Based on our findings, current challenges and further opportunities are discussed. We expect to clarify the feasible theory and functions of RL for building control systems, which would promote their wider-spread application in built environments.