On inexact stochastic splitting methods for a class of nonconvex composite optimization problems with relative error. (2nd January 2023)

• Record Type: Journal Article
• Authors: Hu, Jia; Han, Congying; Guo, Tiande; Zhao, Tong
• Is Part Of: Optimization methods and software. Volume 38:Number 1(2023)
• Page Start: 1
• Page End: 33
• Publication Date: 2023-01-02
• Subjects: Nonsmooth nonconvex optimization -- stochastic approximation -- relative error -- alternating direction method of multipliers -- proximal gradient descent -- variance reduction; 47N10 -- 65K10; Mathematical optimization -- Periodicals; Algorithms -- Periodicals; 519.7
• DOI: 10.1080/10556788.2022.2091562
• Languages: English
• ISSNs: 1055-6788

Abstract

Abstract : We consider minimizing a class of nonconvex composite stochastic optimization problems, and deterministic optimization problems whose objective function consists of an expectation function (or an average of finitely many smooth functions) and a weakly convex but potentially nonsmooth function. And in this paper, we focus on the theoretical properties of two types of stochastic splitting methods for solving these nonconvex optimization problems: stochastic alternating direction method of multipliers and stochastic proximal gradient descent. In particular, several inexact versions of these two types of splitting methods are studied. At each iteration, the proposed schemes inexactly solve their subproblems by using relative error criteria instead of exogenous and diminishing error rules, which allows our approaches to handle some complex regularized problems in statistics and machine learning. Under mild conditions, we obtain the convergence of the schemes and their computational complexity related to the evaluations on the component gradient of the smooth function, and find that some conclusions of their exact counterparts can be recovered.