Average ordinary least squares‐centered penalized regression: A more efficient way to address multicollinearity than ridge regression. (2nd February 2022)

• Record Type: Journal Article
• Authors: Wang, Wei; Li, Linjian; Li, Sheng; Yin, Fei; Liao, Fang; Zhang, Tao; Li, Xiaosong; Xiao, Xiong; Ma, Yue
• Is Part Of: Statistica Neerlandica. Volume 76:Number 3(2022)
• Page Start: 347
• Page End: 368
• Publication Date: 2022-02-02
• Subjects: average OLS‐centered penalized regression -- multicollinearity -- ridge regression -- shrinkage center; Statistics -- Periodicals; 519.5; 314.92
• DOI: 10.1111/stan.12263
• Languages: English
• ISSNs: 0039-0402

Abstract

Abstract: We developed a novel method to address multicollinearity in linear models called average ordinary least squares (OLS)‐centered penalized regression (AOPR). AOPR penalizes the cost function to shrink the estimators toward the weighted‐average OLS estimator. The commonly used ridge regression (RR) shrinks the estimators toward zero, that is, employs penalization prior β ∼ N ( 0, 1 / k ) in the Bayesian view, which contradicts the common real prior β ≠ 0 . Therefore, RR selects small penalization coefficients to relieve such a contradiction and thus makes the penalizations inadequate. Mathematical derivations remind us that AOPR could increase the performance of RR and OLS regression. A simulation study shows that AOPR obtains more accurate estimators than OLS regression in most situations and more accurate estimators than RR when the signs of the true β s are identical and is slightly less accurate than RR when the signs of the true β s are different. Additionally, a case study shows that AOPR obtains more stable estimators and stronger statistical power and predictive ability than RR and OLS regression. Through these results, we recommend using AOPR to address multicollinearity more efficiently than RR and OLS regression, especially when the true β s have identical signs.