SLiKER: Sparse loss induced kernel ensemble regression. (January 2021)

• Record Type: Journal Article
• Authors: Shen, Xiang-Jun; Ni, ChengGong; Wang, Liangjun; Zha, Zheng-Jun
• Is Part Of: Pattern recognition. Volume 109(2021)
• Publication Date: 2021-01
• Subjects: Multiple kernels -- Ensemble regression -- Sparse loss -- Classification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107587
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We develop a novel regression method based on kernel trick and ensemble principle. Its merit is that multi-kernel selection and parameter decision can be conducted automatically through a pool of kernels. In our proposed method, we introduce sparsity to evaluate the quality of the model. With this sparsity model, well-behaved regressors are selected and the impacts of badly-behaved regressors are decreased. Experimental results on UCI regression and computer vision datasets indicate that compared to other regression ensemble methods, such as random forest and XGBoost, our method has the advantages of best performances in keeping lowest regression loss and highest classification accuracy. Abstract: Kernel ridge regression (KRR) is an efficient method for regression task. However, KRR has a deficiency in finding appropriate type of kernel functions and their parameters. To overcome this shortcoming, a novel kernel ensemble framework is developed. In this ensemble framework, each kernel regressor is associated with a weight that can be adaptively determined according to its contribution to the regression result. By this way, more appropriate kernels and more accurate parameters can be learned directly from data without any manual intervention, which results in better performance in regression. In addition, to overcome the influence of existing outliers, the regressor loss is modeled as a sparse signal, thus a Sparse Loss induced Kernel Ensemble Regression (SLiKER) method is obtained. Experimental results on several UCI regression and computer vision datasets show that our proposed approach obtains best regression and classification performances among the state-of-art comparative methods.