An analysis of heuristic metrics for classifier ensemble pruning based on ordered aggregation. (April 2022)

• Record Type: Journal Article
• Authors: Mohammed, Amgad M.; Onieva, Enrique; Woźniak, Michał; Martínez-Muñoz, Gonzalo
• Is Part Of: Pattern recognition. Volume 124(2022)
• Publication Date: 2022-04
• Subjects: Heuristic optimization -- Ensemble selection -- Ensemble pruning -- Classifier ensemble -- Machine learning -- Difficult samples -- Ordering-based pruning -- Classifier complementariness; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108493
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: An up-to-date revision and analysis of ordering-based ensemble pruning methods. Analysis of how the pruning accuracy is affected by the size of the original ensemble. The accuracy of pruned ensembles is shown to be superior to the stable predictions given by bagged ensembles. It is shown how the pruning metrics perform in binary and multiclass classification tasks. A thorough analysis of the prediction consistency, time and space complexities of pruning methods is performed. Abstract: Classifier ensemble pruning is a strategy through which a subensemble can be identified via optimizing a predefined performance criterion. Choosing the optimum or suboptimum subensemble decreases the initial ensemble size and increases its predictive performance. In this article, a set of heuristic metrics will be analyzed to guide the pruning process. The analyzed metrics are based on modifying the order of the classifiers in the bagging algorithm, with selecting the first set in the queue. Some of these criteria include general accuracy, the complementarity of decisions, ensemble diversity, the margin of samples, minimum redundancy, discriminant classifiers, and margin hybrid diversity. The efficacy of those metrics is affected by the original ensemble size, the required subensemble size, the kind of individual classifiers, and the number of classes. While the efficiency is measured in terms of the computational cost and the memory space requirements. The performance of those metrics is assessed over fifteen binary and fifteen multiclass benchmark classification tasks, respectively. In addition, the behavior of those metrics against randomness is measured in terms of the distribution of their accuracy around the median. Results show that ordered aggregation is an efficient strategy to generate subensembles that improve both predictive performance as well as computational and memory complexities of the whole bagging ensemble.