Influence Diagnostics for High-Dimensional Lasso Regression. Issue 4 (2nd October 2019)

• Record Type: Journal Article
• Authors: Rajaratnam, Bala; Roberts, Steven; Sparks, Doug; Yu, Honglin
• Is Part Of: Journal of computational and graphical statistics. Volume 28:Issue 4(2019)
• Page Start: 877
• Page End: 890
• Publication Date: 2019-10-02
• Subjects: Large p small n -- Model selection -- Regression diagnostics -- Shrinkage; Mathematical statistics -- Data processing -- Periodicals; Mathematical statistics -- Graphic methods -- Periodicals; 519.50285
• DOI: 10.1080/10618600.2019.1598869
• Languages: English
• ISSNs: 1061-8600

Abstract

Abstract: The increased availability of high-dimensional data, and appeal of a "sparse" solution has made penalized likelihood methods commonplace. Arguably the most widely utilized of these methods is ℓ 1 regularization, popularly known as the lasso. When the lasso is applied to high-dimensional data, observations are relatively few; thus, each observation can potentially have tremendous influence on model selection and inference. Hence, a natural question in this context is the identification and assessment of influential observations. We address this by extending the framework for assessing estimation influence in traditional linear regression, and demonstrate that it is equally, if not more, relevant for assessing model selection influence for high-dimensional lasso regression. Within this framework, we propose four new "deletion methods" for gauging the influence of an observation on lasso model selection: df-model, df-regpath, df-cvpath, and df-lambda. Asymptotic cut-offs for each measure, even when p → ∞, are developed. We illustrate that in high-dimensional settings, individual observations can have a tremendous impact on lasso model selection. We demonstrate that application of our measures can help reveal relationships in high-dimensional real data that may otherwise remain hidden. Supplementary materials for this article are available online.