A family of subgradient-based methods for convex optimization problems in a unifying framework. (2nd September 2016)

• Record Type: Journal Article
• Authors: Ito, Masaru; Fukuda, Mituhiro
• Is Part Of: Optimization methods and software. Volume 31:Number 5(2016)
• Page Start: 952
• Page End: 982
• Publication Date: 2016-09-02
• Subjects: non-smooth/smooth convex optimization -- structured convex optimization -- subgradient/gradient-based proximal method -- mirror-descent method -- dual-averaging method -- complexity bounds; 90C25 -- 68Q25 -- 49M37; Mathematical optimization -- Periodicals; Algorithms -- Periodicals; 519.7
• DOI: 10.1080/10556788.2016.1182165
• Languages: English
• ISSNs: 1055-6788

Abstract

Abstract : We propose a new family of subgradient- and gradient-based methods which converges with optimal complexity for convex optimization problems whose feasible region is simple enough. This includes cases where the objective function is non-smooth, smooth, have composite/saddle structure, or are given by an inexact oracle model. We unified the way of constructing the subproblems which are necessary to be solved at each iteration of these methods. This permitted us to analyse the convergence of these methods in a unified way compared to previous results which required different approaches for each method/algorithm. Our contribution rely on two well-known methods in non-smooth convex optimization: the mirror-descent method (MDM) by Nemirovski-Yudin and the dual-averaging method by Nesterov. Therefore, our family of methods includes them and many other methods as particular cases. For instance, the proposed family of classical gradient methods and its accelerations generalize Devolder et al.'s, Nesterov's primal/dual gradient methods, and Tseng's accelerated proximal gradient methods. Also our family of methods can partially become special cases of other universal methods, too. As an additional contribution, the novel extended MDM removes the compactness assumption of the feasible region and the fixation of the total number of iterations which is required by the original MDM in order to attain the optimal complexity.