Laplace Error Penalty‐based Variable Selection in High Dimension. (3rd November 2014)

• Record Type: Journal Article
• Authors: Wen, Canhong; Wang, Xueqin; Wang, Shaoli
• Is Part Of: Scandinavian journal of statistics. Volume 42:Number 3(2015:Sep.)
• Page Start: 685
• Page End: 700
• Publication Date: 2014-11-03
• Subjects: Statistics -- Periodicals; 310
• DOI: 10.1111/sjos.12130
• Languages: English
• ISSNs: 0303-6898

Abstract

<abstract abstract-type="main" id="sjos12130-abs-0001"> <title>Abstract</title> <p id="sjos12130-para-0001">We propose the Laplace Error Penalty (LEP) function for variable selection in high‐dimensional regression. Unlike penalty functions using piecewise splines construction, the LEP is constructed as an exponential function with two tuning parameters and is infinitely differentiable everywhere except at the origin. With this construction, the LEP‐based procedure acquires extra flexibility in variable selection, admits a unified derivative formula in optimization and is able to approximate the <italic>L</italic>₀ penalty as close as possible. We show that the LEP procedure can identify relevant predictors in exponentially high‐dimensional regression with normal errors. We also establish the oracle property for the LEP estimator. Although not being convex, the LEP yields a convex penalized least squares function under mild conditions if <italic>p</italic> is no greater than <italic>n</italic>. A coordinate descent majorization‐minimization algorithm is introduced to implement the LEP procedure. In simulations and a real data analysis, the LEP methodology performs favorably among competitive procedures.</p> </abstract>