On the discrepancy principle for stochastic gradient descent. (2nd September 2020)

• Record Type: Journal Article
• Authors: Jahn, Tim; Jin, Bangti
• Is Part Of: Inverse problems. Volume 36:Number 9(2020)
• Publication Date: 2020-09-02
• Subjects: stochastic gradient descent -- discrepancy principle -- convergence; Inverse problems (Differential equations) -- Periodicals; 515.357
• DOI: 10.1088/1361-6420/abaa58
• Languages: English
• ISSNs: 0266-5611

Abstract

Abstract: Stochastic gradient descent (SGD) is a promising numerical method for solving large-scale inverse problems. However, its theoretical properties remain largely underexplored in the lens of classical regularization theory. In this note, we study the classical discrepancy principle, one of the most popular a posteriori choice rules, as the stopping criterion for SGD, and prove the finite-iteration termination property and the convergence of the iterate in probability as the noise level tends to zero. The theoretical results are complemented with extensive numerical experiments.