Selection and Fusion of Categorical Predictors with L0-Type Penalties. (October 2015)

• Record Type: Journal Article
• Authors: Oelker, Margret-Ruth; Pößnecker, Wolfgang; Tutz, Gerhard
• Is Part Of: Statistical modelling. Volume 15:Number 5(2015)
• Page Start: 389
• Page End: 410
• Publication Date: 2015-10
• Subjects: adaptive Lasso -- best subset selection -- GLMs -- model selection; Linear models (Statistics) -- Periodicals; Mathematical models -- Periodicals; Modèles linéaires (Statistique) -- Périodiques; Modèles mathématiques -- Périodiques; Modèle statistique; Modèle linéaire; Modélisation statistique; Périodique électronique (Descripteur de forme); Ressource Internet (Descripteur de forme); 519.5011
• DOI: 10.1177/1471082X14553366
• Languages: English
• ISSNs: 1471-082X

Abstract

In regression modelling, categorical covariates have to be coded. Depending on the number of categorical covariates and on the number of levels they have, the number of coefficients can become huge. To reduce the model complexity, coefficients of similar categories should be fused and coefficients of non-influential categories should be set to zero. To this end, Lasso-type penalties on the differences of coefficients are a standard approach. However, the clustering/selection performance of this approach is sometimes poor–especially when the adaptive weights are badly conditioned or not existing. In some situations, there is no incentive to cluster similar categories. To overcome this, aL 0 penalty on the differences of coefficients is proposed, whereby theL 0 'norm' is defined as the number of non-zero entries in a vector. The proposed penalty favours to find clusters of categories that share the same effect on the response variable while the estimation accuracy is comparable to Lasso-type penalties. Numerical experiments within the framework of generalized linear models are promising. For illustration, data on the unemployment rates in Germany is analyzed.