Fractional ridge regression: a fast, interpretable reparameterization of ridge regression. Issue 12 (30th November 2020)

• Record Type: Journal Article
• Authors: Rokem, Ariel; Kay, Kendrick
• Is Part Of: GigaScience. Volume 9:Issue 12(2020)
• Publication Date: 2020-11-30
• Subjects: general linear model -- hyperparameters -- brain imaging -- open-source software; Information storage and retrieval systems -- Research -- Periodicals; Biology -- Research -- Periodicals; Medical sciences -- Research -- Periodicals; Database management -- Periodicals; 570.285
• DOI: 10.1093/gigascience/giaa133
• Languages: English
• ISSNs: 2047-217X

Abstract

Abstract: Background: Ridge regression is a regularization technique that penalizes the L2-norm of the coefficients in linear regression. One of the challenges of using ridge regression is the need to set a hyperparameter (α) that controls the amount of regularization. Cross-validation is typically used to select the best α from a set of candidates. However, efficient and appropriate selection of α can be challenging. This becomes prohibitive when large amounts of data are analyzed. Because the selected α depends on the scale of the data and correlations across predictors, it is also not straightforwardly interpretable. Results: The present work addresses these challenges through a novel approach to ridge regression. We propose to reparameterize ridge regression in terms of the ratio γ between the L2-norms of the regularized and unregularized coefficients. We provide an algorithm that efficiently implements this approach, called fractional ridge regression, as well as open-source software implementations in Python and matlab (https://github.com/nrdg/fracridge ). We show that the proposed method is fast and scalable for large-scale data problems. In brain imaging data, we demonstrate that this approach delivers results that are straightforward to interpret and compare across models and datasets. Conclusion: Fractional ridge regression has several benefits: the solutions obtained for different γ are guaranteed to vary, guarding against wasted calculations; and automatically span the relevant range of regularization, avoiding the need for arduous manual exploration. These properties make fractional ridge regression particularly suitable for analysis of large complex datasets.