Adaptive regularization for Lasso models in the context of nonstationary data streams. (16th July 2018)

• Record Type: Journal Article
• Authors: Monti, Ricardo P.; Anagnostopoulos, Christoforos; Montana, Giovanni
• Is Part Of: Statistical analysis and data mining. Volume 11:Number 5(2018)
• Page Start: 237
• Page End: 247
• Publication Date: 2018-07-16
• Subjects: adaptive filtering -- ℓ1 regularization -- nonstationary data streams -- time‐varying sparsity; Data mining -- Statistical methods -- Periodicals; 006.312
• DOI: 10.1002/sam.11390
• Languages: English
• ISSNs: 1932-1864

Abstract

Abstract : Large‐scale, streaming data sets are ubiquitous in modern machine learning. Streaming algorithms must be scalable, amenable to incremental training, and robust to the presence of nonstationarity. In this work we consider the problem of learning ℓ 1 regularized linear models in the context of streaming data. In particular, the focus of this work revolves around how to select the regularization parameter when data arrives sequentially and the underlying distribution is nonstationary (implying the choice of optimal regularization parameter is itself time‐varying). We propose a framework through which to infer an adaptive regularization parameter. Our approach employs an ℓ 1 penalty constraint where the corresponding sparsity parameter is iteratively updated via stochastic gradient descent. This serves to reformulate the choice of regularization parameter in a principled framework for online learning. The proposed method is derived for linear regression and subsequently extended to generalized linear models. We validate our approach using simulated and real data sets, concluding with an application to a neuroimaging data set.