On free energy barriers in Gaussian priors and failure of cold start MCMC for high-dimensional unimodal distributions. (15th May 2023)

• Record Type: Journal Article
• Authors: Bandeira, Afonso S.; Maillard, Antoine; Nickl, Richard; Wang, Sven
• Is Part Of: Philosophical transactions. Volume 381:Number 2247(2023)
• Publication Date: 2023-05-15
• Subjects: MCMC -- Bayesian inference -- Gaussian processes -- computational hardness; Physical sciences -- Periodicals; Engineering -- Periodicals; Mathematics -- Periodicals; 500
• DOI: 10.1098/rsta.2022.0150
• Languages: English
• ISSNs: 1364-503X

Abstract

Abstract : We exhibit examples of high-dimensional unimodal posterior distributions arising in nonlinear regression models with Gaussian process priors for which Markov chain Monte Carlo (MCMC) methods can take an exponential run-time to enter the regions where the bulk of the posterior measure concentrates. Our results apply to worst-case initialized ('cold start') algorithms that are local in the sense that their step sizes cannot be too large on average. The counter-examples hold for general MCMC schemes based on gradient or random walk steps, and the theory is illustrated for Metropolis–Hastings adjusted methods such as preconditioned Crank–Nicolson and Metropolis-adjusted Langevin algorithm. This article is part of the theme issue 'Bayesian inference: challenges, perspectives, and prospects'.