Weighted classifier ensemble based on quadratic form. Issue 5 (May 2015)

• Record Type: Journal Article
• Authors: Mao, Shasha; Jiao, Licheng; Xiong, Lin; Gou, Shuiping; Chen, Bo; Yeung, Sai-Kit
• Is Part Of: Pattern recognition. Volume 48:Issue 5(2015:May)
• Page Start: 1688
• Page End: 1706
• Publication Date: 2015-05
• Subjects: Ensemble learning -- Weighted classifier ensemble -- Quadratic form; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2014.10.017
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Diversity and accuracy are the two key factors that decide the ensemble generalization error. Constructing a good ensemble method by balancing these two factors is difficult, because increasing diversity is at the cost of reducing accuracy normally. In order to improve the performance of an ensemble while avoiding the difficulty derived of balancing diversity and accuracy, we propose a novel method that weights each classifier in the ensemble by maximizing three different quadratic forms. In this paper, the optimal weight of individual classifiers is obtained by minimizing the ensemble error, rather than analyzing diversity and accuracy. Since it is difficult to minimize the general form of the ensemble error directly, we approximate the error in an objective function subject to two constraints ( ∑ w i = 1 and − 1 < w i < 1 ). Particularly, we introduce an error term with a weight vector w 0, and subtract this error with the quadratic form to obtain our approximated error. This subtraction makes minimizing the approximation form equivalent to maximizing the original quadratic form. Theoretical analysis finds that when the value of the quadratic form is maximized, the error of an ensemble system with the corresponding optimal weight w * will be smallest, especially compared with the ensemble with w 0 . Finally, we demonstrate improved classification performance from the experimental results of an artificial dataset, UCI datasets and PolSAR image data. Highlights: A new weighted classifier ensemble method is proposed. An approximation form of the ensemble error is introduced. An optimal weight vector is sought based on minimizing the approximation form. It is converted into maximizing quadratic forms by invoking a known weight vector. The larger the value of the quadratic form is, the lower the ensemble error is.