Speeding Up MCMC by Efficient Data Subsampling. Issue 526 (3rd April 2019)

• Record Type: Journal Article
• Authors: Quiroz, Matias; Kohn, Robert; Villani, Mattias; Tran, Minh-Ngoc
• Is Part Of: Journal of the American Statistical Association. Volume 114:Issue 526(2019)
• Page Start: 831
• Page End: 843
• Publication Date: 2019-04-03
• Subjects: Bayesian inference -- Big Data -- Block pseudo-marginal -- Correlated pseudo-marginal -- Estimated likelihood -- Survey sampling; Statistics -- Periodicals; Statistics -- Periodicals; Statistiques -- Périodiques; États-Unis -- Statistiques -- Périodiques; 519.5
• DOI: 10.1080/01621459.2018.1448827
• Languages: English
• ISSNs: 0162-1459

Abstract

ABSTRACT: We propose subsampling Markov chain Monte Carlo (MCMC), an MCMC framework where the likelihood function for n observations is estimated from a random subset of m observations. We introduce a highly efficient unbiased estimator of the log-likelihood based on control variates, such that the computing cost is much smaller than that of the full log-likelihood in standard MCMC. The likelihood estimate is bias-corrected and used in two dependent pseudo-marginal algorithms to sample from a perturbed posterior, for which we derive the asymptotic error with respect to n and m, respectively. We propose a practical estimator of the error and show that the error is negligible even for a very small m in our applications. We demonstrate that subsampling MCMC is substantially more efficient than standard MCMC in terms of sampling efficiency for a given computational budget, and that it outperforms other subsampling methods for MCMC proposed in the literature. Supplementary materials for this article are available online.