An EnKF-based scheme to optimize hyper-parameters and features for SVM classifier. (February 2017)

• Record Type: Journal Article
• Authors: Ji, Yingsheng; Chen, Yushu; Fu, Haohuan; Yang, Guangwen
• Is Part Of: Pattern recognition. Volume 62(2017:Feb.)
• Page Start: 202
• Page End: 213
• Publication Date: 2017-02
• Subjects: EnKF -- SVM -- Hyper-parameter optimization -- Feature weighting -- Feature selection; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.08.014
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: The quality of models built by machine learning algorithms mostly depends on the careful tuning of hyper-parameters and feature weights. This paper introduces a novel scheme to optimize hyper-parameters and features by using the Ensemble Kalman Filter (EnKF), which is an iterative optimization algorithm used for high-dimensional nonlinear systems. We build a framework for applying the EnKF method on parameter optimization problems. We propose ensemble evolution to converge to the global optimum. We also optimize the EnKF calculation for large datasets by using the computationally efficient UR decomposition. To demonstrate the performance of our proposed design, we apply our approach for the tuning problem of Support Vector Machines. Experimental results show that the better global optima can be identified by our approach with acceptable computation cost compared to three state-of-the-art Bayesian optimization methods (SMAC, TPE and SPEARMINT). Abstract : Highlights: We adopt the EnKF based scheme to optimize hyper-parameters and features. We build the EnKF based framework for parameter optimization. Multiple ensemble and ensemble evolution are proposed to enhance effects. Householder method is proposed to optimize the EnKF analysis computation. The EnKF scheme outperforms three current Bayesian optimization methods.