An experimental evaluation of mixup regression forests. (1st August 2020)

• Record Type: Journal Article
• Authors: Rodríguez, Juan J.; Juez-Gil, Mario; Arnaiz-González, Álvar; Kuncheva, Ludmila I.
• Is Part Of: Expert systems with applications. Volume 151(2020)
• Publication Date: 2020-08-01
• Subjects: Mixup -- Regression -- Random forest -- Rotation forest; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2020.113376
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: In the mixup method deep learning models are trained using artificial instances. The instances are obtained by combining pairs of instances and their labels. The use of mixup in ensembles of regression trees is proposed and studied. The mixup approach can improve the results of Random Forest and Rotation Forest. Abstract: Over the past few decades, the remarkable prediction capabilities of ensemble methods have been used within a wide range of applications. Maximization of base-model ensemble accuracy and diversity are the keys to the heightened performance of these methods. One way to achieve diversity for training the base models is to generate artificial/synthetic instances for their incorporation with the original instances. Recently, the mixup method was proposed for improving the classification power of deep neural networks (Zhang, Cissé, Dauphin, and Lopez-Paz, 2017). Mixup method generates artificial instances by combining pairs of instances and their labels, these new instances are used for training the neural networks promoting its regularization. In this paper, new regression tree ensembles trained with mixup, which we will refer to as Mixup Regression Forest, are presented and tested. The experimental study with 61 datasets showed that the mixup approach improved the results of both Random Forest and Rotation Forest.