Least squares auto-tuning. Issue 5 (4th May 2021)

• Record Type: Journal Article
• Authors: Barratt, Shane T.; Boyd, Stephen P.
• Is Part Of: Engineering optimization. Volume 53:Issue 5(2021)
• Page Start: 789
• Page End: 810
• Publication Date: 2021-05-04
• Subjects: Least squares -- hyper-parameter optimization -- proximal gradient method; Engineering design -- Periodicals; Mathematical optimization -- Periodicals; 620.0042
• DOI: 10.1080/0305215X.2020.1754406
• Languages: English
• ISSNs: 0305-215X

Abstract

Abstract : Least squares auto-tuning automatically finds hyper-parameters in least squares problems that minimize another (true) objective. The least squares tuning optimization problem is non-convex, so it cannot be solved efficiently. This article presents a powerful proximal gradient method for least squares auto-tuning that can be used to find good, if not the best, hyper-parameters for least squares problems. The application of least squares auto-tuning to data fitting is discussed. Numerical experiments on a classification problem using the MNIST dataset demonstrate the effectiveness of the method; it is able to cut the test error of standard least squares in half.