Inexact basic tensor methods for some classes of convex optimization problems. (4th May 2022)

• Record Type: Journal Article
• Authors: Nesterov, Yurii
• Is Part Of: Optimization methods and software. Volume 37:Number 3(2022)
• Page Start: 878
• Page End: 906
• Publication Date: 2022-05-04
• Subjects: High-order methods -- tensor methods -- complexity bounds -- convex optimization; 90C25; Mathematical optimization -- Periodicals; Algorithms -- Periodicals; 519.7
• DOI: 10.1080/10556788.2020.1854252
• Languages: English
• ISSNs: 1055-6788

Abstract

ABSTRACT: In this paper, we analyse the Basic Tensor Methods, which use approximate solutions of the auxiliary problems. The quality of this solution is described by the residual in the function value, which must be proportional to ϵ p + 1 p, where p ≥ 1 is the order of the method and ϵ is the desired accuracy in the main optimization problem. We analyse in details the auxiliary schemes for the third- and second-order tensor methods. The auxiliary problems for the third-order scheme can be solved very efficiently by a linearly convergent gradient-type method with a preconditioner. The most expensive operation in this process is a preliminary factorization of the Hessian of the objective function. For solving the auxiliary problem for the second order scheme, we suggest two variants of the Fast Gradient Methods with restart, which converge as O ( 1 k 6 ), where k is the iteration counter. Finally, we present the results of the preliminary computational experiments.