A multilevel approach for stochastic nonlinear optimal control. Issue 5 (4th May 2022)

• Record Type: Journal Article
• Authors: Jasra, Ajay; Heng, Jeremy; Xu, Yaxian; Bishop, Adrian N.
• Is Part Of: International journal of control. Volume 95:Issue 5(2022)
• Page Start: 1290
• Page End: 1304
• Publication Date: 2022-05-04
• Subjects: Optimal control -- multilevel Monte Carlo -- Markov chain Monte Carlo -- sequential Monte Carlo; Automatic control -- Periodicals; Electronic journals; 629.8
• DOI: 10.1080/00207179.2020.1849805
• Languages: English
• ISSNs: 0020-7179

Abstract

Abstract : We consider a class of finite-time horizon nonlinear stochastic optimal control problem. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging task. We propose a new Monte Carlo approach that significantly improves upon existing methodology. We tackle the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state-space model, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce the cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with O ( ϵ 2 ) mean squared error with a cost of O ( ϵ − 2 log ⁡ ( ϵ ) 2 ) . In contrast, a cost of O ( ϵ − 3 ) is required for the existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples.