Efficient posterior sampling for high-dimensional imbalanced logistic regression. (17th June 2020)

• Record Type: Journal Article
• Authors: Sen, Deborshee; Sachs, Matthias; Lu, Jianfeng; Dunson, David B
• Is Part Of: Biometrika. Volume 107:Number 4(2020:Dec.)
• Page Start: 1005
• Page End: 1012
• Publication Date: 2020-06-17
• Subjects: Imbalanced data -- Logistic regression -- Piecewise-deterministic Markov process -- Scalable inference -- Subsampling; Biometry -- Periodicals; 570.1519505
• DOI: 10.1093/biomet/asaa035
• Languages: English
• ISSNs: 0006-3444

Abstract

Summary: Classification with high-dimensional data is of widespread interest and often involves dealing with imbalanced data. Bayesian classification approaches are hampered by the fact that current Markov chain Monte Carlo algorithms for posterior computation become inefficient as the number $p$ of predictors or the number $n$ of subjects to classify gets large, because of the increasing computational time per step and worsening mixing rates. One strategy is to employ a gradient-based sampler to improve mixing while using data subsamples to reduce the per-step computational complexity. However, the usual subsampling breaks down when applied to imbalanced data. Instead, we generalize piecewise-deterministic Markov chain Monte Carlo algorithms to include importance-weighted and mini-batch subsampling. These maintain the correct stationary distribution with arbitrarily small subsamples and substantially outperform current competitors. We provide theoretical support for the proposed approach and demonstrate its performance gains in simulated data examples and an application to cancer data.