Purposeful cross-validation: a novel cross-validation strategy for improved surrogate optimizability. Issue 9 (2nd September 2021)

• Record Type: Journal Article
• Authors: Correia, Daniel; Wilke, Daniel N.
• Is Part Of: Engineering optimization. Volume 53:Issue 9(2021)
• Page Start: 1558
• Page End: 1573
• Publication Date: 2021-09-02
• Subjects: Surrogate-based optimization -- cross validation -- radial basis function -- design optimization; Engineering design -- Periodicals; Mathematical optimization -- Periodicals; 620.0042
• DOI: 10.1080/0305215X.2020.1807017
• Languages: English
• ISSNs: 0305-215X

Abstract

ABSTRACT: Parameter selection during the construction of surrogates is often conducted by minimizing the Mean Squared Cross-Validation Error (MSE-CV). Surrogates constructed using MSE are poorly optimized using gradient-based optimizers. Hence, Nelder–Mead like optimizers are often favoured, which is unfortunate as surrogates make analytical gradients freely available and gradient-based optimizers scale better with increasing dimension. To address this shortcoming, this article proposes a new Cross-Validation (CV) approach, by optimizing the surrogate and computing the Mean Optimizer Distance (MOD-CV) to the best design in the surrogate. Four experimental CV measures are compared on seven test problems and it is demonstrated that the performance of gradient-based optimizers can be significantly enhanced, with a possible 97% improvement in MOD-CV over MSE-CV using Sequential Least-Squares Quadratic Programming (SLSQP). Additionally, surrogates constructed using MOD-CV outperform surrogates constructed with MSE-CV, 80% of the time when optimized with SLSQP and 68% when optimized with Nelder-Mead.