Parallel Variational Bayes for Large Datasets With an Application to Generalized Linear Mixed Models. Issue 2 (2nd April 2016)

• Record Type: Journal Article
• Authors: Tran, Minh-Ngoc; Nott, David J.; Kuk, Anthony Y. C.; Kohn, Robert
• Is Part Of: Journal of computational and graphical statistics. Volume 25:Issue 2(2016)
• Page Start: 626
• Page End: 646
• Publication Date: 2016-04-02
• Subjects: Divide and recombine -- Fixed-form variational Bayes -- Improved variational Bayes approximation -- Mean-field variational Bayes -- Parallelization; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2015.1012293
• Languages: English
• ISSNs: 1061-8600

Abstract

Abstract : The article develops a hybrid variational Bayes (VB) algorithm that combines the mean-field and stochastic linear regression fixed-form VB methods. The new estimation algorithm can be used to approximate any posterior without relying on conjugate priors. We propose a divide and recombine strategy for the analysis of large datasets, which partitions a large dataset into smaller subsets and then combines the variational distributions that have been learned in parallel on each separate subset using the hybrid VB algorithm. We also describe an efficient model selection strategy using cross-validation, which is straightforward to implement as a by-product of the parallel run. The proposed method is applied to fitting generalized linear mixed models. The computational efficiency of the parallel and hybrid VB algorithm is demonstrated on several simulated and real datasets. Supplementary material for this article is available online.