Efficient learning ensemble SuperParent-one-dependence estimator by maximizing conditional log likelihood. Issue 21 (30th November 2015)

• Record Type: Journal Article
• Authors: Zheng, Xiaolin; Lin, Zhen; Xu, Huan; Chen, Chaochao; Ye, Ting
• Is Part Of: Expert systems with applications. Volume 42:Issue 21(2015)
• Page Start: 7732
• Page End: 7745
• Publication Date: 2015-11-30
• Subjects: Classification -- Gradient methods -- Machine learning -- Modeling structured -- Conditional likelihood; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2015.05.051
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Existing shortages of main ensemble methodologies are studied. A novel ensemble SPODE algorithm by maximizing CLL is proposed. SGD iterative method is used to train the weight parameters. Analyses show EODE-CLL improves the performance of the entire ensemble classifiers. Experiments of EODE-CLL outperform significantly than state-of-the-art methods. Abstract: The ensemble of SuperParent one-dependence estimators (SPODEs) is one of the most effective improved algorithms. It achieves high classification accuracy while decreasing variance. However, most existing approaches only focus on performance improvement of individual SPODEs in selection and weighting procedures but overlook the importance of the entire ensemble model. Based on the assumption that the performance of the entire ensemble classifier can obtain better weight distribution than using the greedy strategy inside each SPODE, we propose an ensemble SPODE algorithm by maximizing conditional log likelihood (EODE-CLL). First, we choose the maximum conditional probability as the global optimization goal, which can avoid over-fitting problem compared with the least squares error. Second, the algorithm assigns hierarchical weights for SPODEs and the attributes inside SPODE. The second weight layer can help fully optimize local SPODE model. Finally, stochastic gradient descent method is used to search best parameters. It has good scalability, which has spawned batch and distributed version. Compared to the existing ensemble SPODEs, our proposed model achieves more accurate and robust classification results, while shows better time complexity. We conduct experiments on a public benchmark containing 36 datasets. The results of the experiments show that our EODE-CLL significantly outperforms state-of-the-art ensemble SPODE methods in terms of accuracy, F-measure, bias, and variance.