Heterogeneous oblique random forest. (March 2020)

• Record Type: Journal Article
• Authors: Katuwal, Rakesh; Suganthan, P.N.; Zhang, Le
• Is Part Of: Pattern recognition. Volume 99(2020:Mar.)
• Publication Date: 2020-03
• Subjects: Benchmarking -- Classifiers -- Oblique random forest -- Heterogeneous -- One-vs-all -- Ensemble learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107078
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a heterogeneous oblique random forest that employ a linear (oblique) hyperplane at each node. The hyperplanes are obtained via linear classifiers trained on selected one-vs-all and hyperclasses based partitions. The selection of an oblique hyperplane at each node is based on the optimization of an impurity criterion. The heterogeneous oblique decision trees are more accurate and diverse than other standard decision tree variants. On benchmarking 190 classifiers on 121 UCI datasets, the proposed oblique random forests are the top 3 ranked classifiers. Abstract: Decision trees in random forests use a single feature in non-leaf nodes to split the data. Such splitting results in axis-parallel decision boundaries which may fail to exploit the geometric structure in the data. In oblique decision trees, an oblique hyperplane is employed instead of an axis-parallel hyperplane. Trees with such hyperplanes can better exploit the geometric structure to increase the accuracy of the trees and reduce the depth. The present realizations of oblique decision trees do not evaluate many promising oblique splits to select the best. In this paper, we propose a random forest of heterogeneous oblique decision trees that employ several linear classifiers at each non-leaf node on some top ranked partitions which are obtained via one-vs-all and two-hyperclasses based approaches and ranked based on ideal Gini scores and cluster separability. The oblique hyperplane that optimizes the impurity criterion is then selected as the splitting hyperplane for that node. We benchmark 190 classifiers on 121 UCI datasets. The results show that the oblique random forests proposed in this paper are the top 3 ranked classifiers with the heterogeneous oblique random forest being statistically better than all 189 classifiers in the literature.