Discussion: Deterioration of performance of the lasso with many predictors. (June 2016)

• Record Type: Journal Article
• Authors: Flynn, Cheryl J.; Hurvich, Clifford M.; Simonoff, Jeffrey S.
• Is Part Of: Statistical modelling. Volume 16:Number 3(2016)
• Page Start: 212
• Page End: 216
• Publication Date: 2016-06
• Subjects: deterioration -- Oracle inequalities; Linear models (Statistics) -- Periodicals; Mathematical models -- Periodicals; Modèles linéaires (Statistique) -- Périodiques; Modèles mathématiques -- Périodiques; Modèle statistique; Modèle linéaire; Modélisation statistique; Périodique électronique (Descripteur de forme); Ressource Internet (Descripteur de forme); 519.5011
• DOI: 10.1177/1471082X16642643
• Languages: English
• ISSNs: 1471-082X

Abstract

Oracle inequalities provide probability loss bounds for the lasso estimator at a deterministic choice of the regularization parameter and are commonly cited as theoretical justification for the lasso and its ability to handle high-dimensional settings. Unfortunately, in practice, the regularization parameter is not selected to be a deterministic quantity, but is instead chosen using a random, data-dependent procedure, often making these inequalities misleading in their implications. We discuss general results and demonstrate empirically for data using categorical predictors that the amount of deterioration in performance of the lasso as the number of unnecessary predictors increases can be far worse than the oracle inequalities suggest, but imposing structure on the form of the estimates can reduce this deterioration substantially.