A novel combining classifier method based on Variational Inference. (January 2016)

• Record Type: Journal Article
• Authors: Nguyen, Tien Thanh; Nguyen, Thi Thu Thuy; Pham, Xuan Cuong; Liew, Alan Wee-Chung
• Is Part Of: Pattern recognition. Volume 49(2016:Jan.)
• Page Start: 198
• Page End: 212
• Publication Date: 2016-01
• Subjects: Ensemble method -- Multi classifier system -- Mixture of experts -- Classifier fusion -- Combining classifier algorithm -- Variational Inference -- Multivariate Gaussian distribution; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2015.06.016
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: In this paper, we propose a combining classifier method based on the Bayesian inference framework. Specifically, the outputs of base classifiers (called Level1 data or meta-data) are utilized in a combiner to produce the final classification. In our ensemble system, each class in the training set induces a distribution on the Level1 data, which is modeled by a multivariate Gaussian distribution. Traditionally, the parameters of the Gaussian are estimated using a maximum likelihood approach. However, maximum likelihood estimation cannot be applied since the covariance matrix of Level1 data of each class is not full rank. Instead, we propose to estimate the multivariate Gaussian distribution of Level1 data of each class by using the Variational Inference method. Experiments conducted on eighteen UCI Machine Learning Repository datasets and a selected 10-class CLEF2009 medical imaging database demonstrated the advantage of our method compared with several well-known ensemble methods. Highlights: We proposed a Variational Inference method for multivariate Gaussian distribution estimation. We proposed a novel combining classifier method by employing Variational Inference on base classifiers' outputs. The proposed method is highly competitive to many state-of-the-arts ensemble classifier methods.