Optimal PID and Antiwindup Control Design as a Reinforcement Learning Problem. Issue 2 (2020)

• Record Type: Journal Article
• Authors: Lawrence, Nathan P.; Stewart, Gregory E.; Loewen, Philip D.; Forbes, Michael G.; Backstrom, Johan U.; Gopaluni, R. Bhushan
• Is Part Of: IFAC-PapersOnLine. Volume 53:Issue 2(2020)
• Page Start: 236
• Page End: 241
• Publication Date: 2020
• Subjects: neural networks -- reinforcement learning -- actor-critic networks -- process control -- PID control -- anti-windup compensation; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2020.12.129
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: Deep reinforcement learning (DRL) has seen several successful applications to process control. Common methods rely on a deep neural network structure to model the controller or process. With increasingly complicated control structures, the closed-loop stability of such methods becomes less clear. In this work, we focus on the interpretability of DRL control methods. In particular, we view linear fixed-structure controllers as shallow neural networks embedded in the actor-critic framework. PID controllers guide our development due to their simplicity and acceptance in industrial practice. We then consider input saturation, leading to a simple nonlinear control structure. In order to effectively operate within the actuator limits we then incorporate a tuning parameter for anti-windup compensation. Finally, the simplicity of the controller allows for straightforward initialization. This makes our method inherently stabilizing, both during and after training, and amenable to known operational PID gains.