Calibration of ϵ−insensitive loss in support vector machines regression. Issue 4 (March 2019)

• Record Type: Journal Article
• Authors: Tong, Hongzhi; Ng, Michael K.
• Is Part Of: Journal of the Franklin Institute. Volume 356:Issue 4(2019)
• Page Start: 2111
• Page End: 2129
• Publication Date: 2019-03
• Subjects: Science -- Periodicals; Technology -- Periodicals; Patents -- United States -- Periodicals; 505
• DOI: 10.1016/j.jfranklin.2018.11.021
• Languages: English
• ISSNs: 0016-0032

Abstract

Abstract: Support vector machines regression (SVMR) is an important tool in many machine learning applications. In this paper, we focus on the theoretical understanding of SVMR based on the ϵ − insensitive loss. For fixed ϵ ≥ 0 and general data generating distributions, we show that the minimizer of the expected risk for ϵ − insensitive loss used in SVMR is a set-valued function called conditional ϵ − median. We then establish a calibration inequality of ϵ − insensitive loss under a noise condition on the conditional distributions. This inequality also ensures us to present a nontrivial variance-expectation bound for ϵ − insensitive loss, and which is known to be important in statistical analysis of the regularized learning algorithms. With the help of the calibration inequality and variance-expectation bound, we finally derive an explicit learning rate for SVMR in some L r − space.