Safe exploration in model-based reinforcement learning using control barrier functions. (January 2023)

• Record Type: Journal Article
• Authors: Cohen, Max H.; Belta, Calin
• Is Part Of: Automatica. Volume 147(2023)
• Publication Date: 2023-01
• Subjects: Adaptive control -- Control barrier functions -- Reinforcement learning; Automatic control -- Periodicals; Automation -- Periodicals; 629.805
• DOI: 10.1016/j.automatica.2022.110684
• Languages: English
• ISSNs: 0005-1098

Abstract

Abstract: This paper develops a model-based reinforcement learning (MBRL) framework for learning online the value function of an infinite-horizon optimal control problem while obeying safety constraints expressed as control barrier functions (CBFs). Our approach is facilitated by the development of a novel class of CBFs, termed Lyapunov-like CBFs (LCBFs), that retain the beneficial properties of CBFs for developing minimally-invasive safe control policies while also possessing desirable Lyapunov-like qualities such as positive semi-definiteness. We show how these LCBFs can be used to augment a learning-based control policy to guarantee safety and then leverage this approach to develop a safe exploration framework in a MBRL setting. We demonstrate that our approach can handle more general safety constraints than comparative methods via numerical examples.