A modified limited memory steepest descent method motivated by an inexact super-linear convergence rate analysis. (13th April 2020)

• Record Type: Journal Article
• Authors: Gu, Ran; Du, Qiang
• Is Part Of: IMA journal of numerical analysis. Volume 41:Number 1(2021)
• Page Start: 247
• Page End: 270
• Publication Date: 2020-04-13
• Subjects: Barzilai–Borwein method -- gradient descent -- step size -- R-linear convergence -- limited memory; Numerical analysis -- Periodicals; 519.405
• DOI: 10.1093/imanum/drz059
• Languages: English
• ISSNs: 0272-4979

Abstract

Abstract: How to choose the step size of gradient descent method has been a popular subject of research. In this paper we propose a modified limited memory steepest descent method (MLMSD). In each iteration we propose a selection rule to pick a unique step size from a candidate set, which is calculated by Fletcher's limited memory steepest descent method (LMSD), instead of going through all the step sizes in a sweep, as in Fletcher's original LMSD algorithm. MLMSD is motivated by an inexact super-linear convergence rate analysis. The R-linear convergence of MLMSD is proved for a strictly convex quadratic minimization problem. Numerical tests are presented to show that our algorithm is efficient and robust.