Optimization of the Model Predictive Control Update Interval Using Reinforcement Learning⁎This work was financed by grants from the Research Council of Norway (PhD Scholarships at SINTEF grant no. 272402, and NTNU AMOS grant No. 223254). Issue 14 (2021)

• Record Type: Journal Article
• Authors: Bøhn, Eivind; Gros, Sebastien; Moe, Signe; Johansen, Tor Arne
• Is Part Of: IFAC-PapersOnLine. Volume 54:Issue 14(2021)
• Page Start: 257
• Page End: 262
• Publication Date: 2021
• Subjects: model predictive control -- reinforcement learning -- event-driven control; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2021.10.362
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: In control applications there is often a compromise that needs to be made with respect to the complexity and performance of the controller, and the computational resources that are available. For instance, the typical hardware platform in embedded control applications is a microcontroller with limited memory and processing power, and for battery powered applications the control system can account for a significant portion of the energy consumption. We propose a controller architecture in which the computational cost is explicitly optimized along with the control objective. This is achieved by a three-part architecture where a high-level, computationally expensive controller generates plans, which a computationally simpler controller executes by compensating for prediction errors, while a recomputation policy decides when the plan should be recomputed. In this paper, we employ model predictive control (MPC) as the high-level plan-generating controller, a linear state feedback controller as the simpler compensating controller, and reinforcement learning (RL) to learn the recomputation policy. Simulation results for the classic control task of balancing an inverted pendulum show that not only is the total processor time reduced by 60% — the RL policy is even able to uncover a non-trivial synergistic relationship between the MPC and the state feedback controller - improving the control performance by 20% over the MPC alone.