Tree regression models using statistical testing and mixed integer programming. (March 2021)

• Record Type: Journal Article
• Authors: Gkioulekas, Ioannis; Papageorgiou, Lazaros G.
• Is Part Of: Computers & industrial engineering. Volume 153(2021)
• Publication Date: 2021-03
• Subjects: Mathematical programming -- Regression analysis -- Decision trees -- Subset selection -- Optimisation; Engineering -- Data processing -- Periodicals; Industrial engineering -- Periodicals; 620.00285
• DOI: 10.1016/j.cie.2020.107059
• Languages: English
• ISSNs: 0360-8352

Abstract

Highlights: Tree regression algorithm using a statistical test to control the size of the tree. Formulation of an optimal subset selection model using information criteria. Integration of subset selection model to the algorithm for node splitting. Competitive performance against established literature tree regression algorithms. Abstract: Regression analysis is a statistical procedure that fits a mathematical function to a set of data in order to capture the relationship between dependent and independent variables. In tree regression, tree structures are constructed by repeated splits of the input space into two subsets, creating if-then-else rules. Such models are popular in the literature due to their ability to be computed quickly and their simple interpretations. This work introduces a tree regression algorithm that exploits an optimisation model of an existing literature method called Mathematical Programming Tree (MPtree ) to optimally split nodes into subsets and applies a statistical test to assess the quality of the partitioning. Additionally, an approach of splitting nodes using multivariate decision rules is explored in this work and compared in terms of performance and computational efficiency. Finally, a novel mathematical model is introduced that performs subset selection on each node in order to select an optimal set of variables to considered for splitting, that improves the computational performance of the proposed algorithm.