Stochastic proximal quasi-Newton methods for non-convex composite optimization. (3rd September 2019)

• Record Type: Journal Article
• Authors: Wang, Xiaoyu; Wang, Xiao; Yuan, Ya-xiang
• Is Part Of: Optimization methods and software. Volume 34:Number 5(2019)
• Page Start: 922
• Page End: 948
• Publication Date: 2019-09-03
• Subjects: Non-convex composite optimization -- Polyak-Łojasiewicz (PL) inequality -- stochastic gradient -- stochastic variance reduction gradient -- symmetric rank one method -- rank one proximity operator -- complexity bound; 47N10 -- 65K10; Mathematical optimization -- Periodicals; Algorithms -- Periodicals; 519.7
• DOI: 10.1080/10556788.2018.1471141
• Languages: English
• ISSNs: 1055-6788

Abstract

ABSTRACT: In this paper, we propose a generic algorithmic framework for stochastic proximal quasi-Newton (SPQN) methods to solve non-convex composite optimization problems. Stochastic second-order information is explored to construct proximal subproblem. Under mild conditions we show the non-asympotic convergence of the proposed algorithm to stationary point of original problems and analyse its computational complexity. Besides, we extend the proximal form of Polyak-Łojasiewicz (PL) inequality to constrained settings and obtain the constrained proximal PL (CP-PL) inequality. Under CP-PL inequality linear convergence rate of the proposed algorithm is achieved. Moreover, we propose a modified self-scaling symmetric rank one incorporated in the framework for SPQN method, which is called stochastic symmetric rank one method. Finally, we report some numerical experiments to reveal the effectiveness of the proposed algorithm.