An active-set proximal quasi-Newton algorithm for ℓ1-regularized minimization over a sphere constraint. (16th December 2022)

• Record Type: Journal Article
• Authors: Shen, Chungen; Mi, Ling; Zhang, Lei-Hong
• Is Part Of: Optimization. Volume 71:Number 16(2022)
• Page Start: 4623
• Page End: 4664
• Publication Date: 2022-12-16
• Subjects: ℓ1-regularized optimization -- spherical constraint -- proximal gradient method -- quasi-Newton method -- active-set method; 90C30; Mathematical optimization -- Periodicals; 519.7
• DOI: 10.1080/02331934.2021.1958809
• Languages: English
• ISSNs: 0233-1934

Abstract

Abstract : The ℓ 1 -regularized minimization has been widely used in many data science applications, and certain special constrained ℓ 1 -regularized minimizations have also been proposed in some recent applications. In this paper, we consider a sphere constrained ℓ 1 -regularized minimization, which can arise in image processing, signal recognition and sparse principal component analysis. Viewing the sphere as a simple Riemannian manifold, manifold-based methods for non-smooth minimization can be applied to solve such a problem, but may still encounter slow convergence in some situations. Our objective of this paper is to propose a new and efficient active-set proximal quasi-Newton method for this problem. The idea behind is to speed up the convergence by separately handling the convergence of both the active and inactive variables. In particular, our method invokes a procedure to effectively estimate active and inactive variables, and then designs the search directions based on proximal gradients and quasi-Newton directions to efficiently treat the convergence of the active and inactive variables, respectively. We show that under some mild conditions, the global convergence is guaranteed, and the complexity is also performed to reveal the computational efficiency. Numerical experience on the ℓ 1 -regularized quadratic programming and sparse principal component analysis on both synthetic and real data demonstrates its robustness and efficiency.