A note on the interpretation of tree‐based regression models. Issue 6 (25th May 2020)

• Record Type: Journal Article
• Authors: Gottard, Anna; Vannucci, Giulia; Marchetti, Giovanni Maria
• Is Part Of: Biometrical journal. Volume 62:Issue 6(2020:Oct.)
• Page Start: 1564
• Page End: 1573
• Publication Date: 2020-05-25
• Subjects: interpretable machine learning -- marginal and conditional dependence -- underlying explanatory process -- variable importance -- variable selection bias; Biometry -- Periodicals; Medical statistics -- Periodicals; 570.15195
• DOI: 10.1002/bimj.201900195
• Languages: English
• ISSNs: 0323-3847

Abstract

Abstract: Tree‐based models are a popular tool for predicting a response given a set of explanatory variables when the regression function is characterized by a certain degree of complexity. Sometimes, they are also used to identify important variables and for variable selection. We show that if the generating model contains chains of direct and indirect effects, then the typical variable importance measures suggest selecting as important mainly the background variables, which have a strong indirect effect, disregarding the variables that directly influence the response. This is attributable mainly to the variable choice in the first steps of the algorithm selecting the splitting variable and to the greedy nature of such search. This pitfall could be relevant when using tree‐based algorithms for understanding the underlying generating process, for population segmentation and for causal inference.