Coordinate descent with arbitrary sampling II: expected separable overapproximation. (2nd September 2016)

• Record Type: Journal Article
• Authors: Qu, Zheng; Richtárik, Peter
• Is Part Of: Optimization methods and software. Volume 31:Number 5(2016)
• Page Start: 858
• Page End: 884
• Publication Date: 2016-09-02
• Subjects: coordinate descent -- randomized methods -- arbitrary sampling -- expected separable overapproximation -- parallel and distributed coordinate descent; Mathematical optimization -- Periodicals; Algorithms -- Periodicals; 519.7
• DOI: 10.1080/10556788.2016.1190361
• Languages: English
• ISSNs: 1055-6788

Abstract

Abstract : The design and complexity analysis of randomized coordinate descent methods, and in particular of variants which update a random subset (sampling) of coordinates in each iteration, depend on the notion of expected separable overapproximation (ESO). This refers to an inequality involving the objective function and the sampling, capturing in a compact way certain smoothness properties of the function in a random subspace spanned by the sampled coordinates. ESO inequalities were previously established for special classes of samplings only, almost invariably for uniform samplings. In this paper we develop a systematic technique for deriving these inequalities for a large class of functions and for arbitrary samplings . We demonstrate that one can recover existing ESO results using our general approach, which is based on the study of eigenvalues associated with samplings and the data describing the function.