Multiple strong and balanced cluster-based ensemble of deep learners. (November 2020)

• Record Type: Journal Article
• Authors: Jan, Zohaib; Verma, Brijesh
• Is Part Of: Pattern recognition. Volume 107(2020:Nov.)
• Publication Date: 2020-11
• Subjects: Deep learning, Ensemble classifier -- Neural networks -- Clustering; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107420
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Random subspace generation through clustering. A methodology of classifying clusters as balanced or strong data clusters. A methodology of balancing strong data clusters. A methodology of training an ensemble of CNNs on small structured non image datasets. Abstract: Convolutional Neural Networks (CNNs), also known as deep learners have seen much success in the last few years due to the availability of large amounts of data and high-performance computational resources. A CNN can be trained effectively if large amounts of data are available as it enables a CNN to find the optimal set of features and weights that can achieve the highest generalization performance. However, due to the requirement of large data size, CNNs require a lot of resources for example running time and computational resources to achieve a reasonable performance. Additionally, unbalanced data makes it difficult to train a CNN effectively that can achieve good generalization performance. In order to alleviate these limitations, in this paper, we propose a novel ensemble of deep learners that learns by combining multiple deep learners trained on small strongly class associated input data effectively. We propose a novel methodology of generating random subspace through clustering input data and propose a measure which can classify each cluster as a strong data cluster and a balanced data cluster. A methodology is also proposed that balances all strong data clusters in the pool so that an architecturally simple CNN can be trained on all balanced data clusters simultaneously. Classification decisions on all trained CNNs are then fused through majority voting to generate class decisions of the ensemble. The performance of the proposed ensemble approach is evaluated on UCI benchmark datasets, and results are compared with existing state-of-the-art ensemble approaches. Significance testing was conducted to further validate the efficacy of the results and a significance test analysis is presented.