Assessment and Adjustment of Approximate Inference Algorithms Using the Law of Total Variance. Issue 4 (2nd October 2021)

• Record Type: Journal Article
• Authors: Yu, Xuejun; Nott, David J.; Tran, Minh-Ngoc; Klein, Nadja
• Is Part Of: Journal of computational and graphical statistics. Volume 30:Issue 4(2021)
• Page Start: 977
• Page End: 990
• Publication Date: 2021-10-02
• Subjects: Deep neural network -- Gaussian process -- Law of total variance -- Likelihood-free inference -- Markov chain Monte Carlo -- Variational approximation; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2021.1880921
• Languages: English
• ISSNs: 1061-8600

Abstract

Abstract: A common method for assessing validity of Bayesian sampling or approximate inference methods makes use of simulated data replicates for parameters drawn from the prior. Under continuity assumptions, quantiles of functions of the simulated parameter values for corresponding posterior distributions are uniformly distributed. Checking for uniformity when a posterior density is approximated numerically provides a diagnostic for algorithm validity. Furthermore, adjustments to achieve uniformity can improve the quality of approximate inference methods. The present article develops a moment-based alternative to the conventional checking and adjustment methods using quantiles. The new approach relates prior and posterior expectations and covariances through the tower property of conditional expectation and the law of total variance. For adjustment, approximate inferences are modified so that the correct prior to posterior relationships hold. We illustrate the method in three examples. The first uses an auxiliary model in a likelihood-free inference problem. The second considers corrections for variational Bayes approximations in a deep neural network generalized linear mixed model. Our final application considers a deep neural network surrogate for approximating Gaussian process regression predictive inference. Supplementary files for this article are available online.