A new algorithm for support vector regression with automatic selection of hyperparameters. (January 2023)

• Record Type: Journal Article
• Authors: Wang, You-Gan; Wu, Jinran; Hu, Zhi-Hua; McLachlan, Geoffrey J.
• Is Part Of: Pattern recognition. Volume 133(2023)
• Publication Date: 2023-01
• Subjects: Automatic selection -- Loss functions -- Noise models -- Parameter estimation -- Probability regularization; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108989
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Establishing an extended primal objective function based on probability regularization leading to a data dependent proportion parameter ν and ϵ . This regularization establishes the equivalence of the ϵ -SVR and ν -SVR in which ν is specified as a proportion of ϵ − 1 log ( 1 + ϵ ) . It proposes values that are equivalent to the maximum likelihood estimates under the distributional assumptions by the loss function; ν is an explicit function of ϵ . Abstract: The hyperparameters in support vector regression (SVR) determine the effectiveness of the support vectors with fitting and predictions. However, the choice of these hyperparameters has always been challenging in both theory and practice. The ν -support vector regression eliminates the need to specify an ϵ value elegantly, but at the cost of specifying or postulating a ν value. We propose an extended primal objective function arising from probability regularization leading to an automatic selection of ϵ, and we can express ν as an explicit function of ϵ . The resultant hyperparameter values can be interpreted as 'working' values required only in training but not testing or prediction. This regularized algorithm, namely ϵ * -SVR, automatically provides a data-dependent ϵ and is found to have a close connection to the ν -support vector regression in the sense that ν as a fraction is a sensible function of ϵ . The ϵ * -SVR automatically selects both ν and ϵ values. We illustrate these findings with some public benchmark datasets.