Regularized regression for categorical data. (June 2016)

• Record Type: Journal Article
• Authors: Tutz, Gerhard; Gertheiss, Jan
• Is Part Of: Statistical modelling. Volume 16:Number 3(2016)
• Page Start: 161
• Page End: 200
• Publication Date: 2016-06
• Subjects: boosting -- categorical data -- fused lasso -- group lasso -- multinomial model -- proportional odds model -- regression trees; Linear models (Statistics) -- Periodicals; Mathematical models -- Periodicals; Modèles linéaires (Statistique) -- Périodiques; Modèles mathématiques -- Périodiques; Modèle statistique; Modèle linéaire; Modélisation statistique; Périodique électronique (Descripteur de forme); Ressource Internet (Descripteur de forme); 519.5011
• DOI: 10.1177/1471082X16642560
• Languages: English
• ISSNs: 1471-082X

Abstract

In the last two decades, regularization techniques, in particular penalty-based methods, have become very popular in statistical modelling. Driven by technological developments, most approaches have been designed for high-dimensional problems with metric variables, whereas categorical data has largely been neglected. In recent years, however, it has become clear that regularization is also very promising when modelling categorical data. A specific trait of categorical data is that many parameters are typically needed to model the underlying structure. This results in complex estimation problems that call for structured penalties which are tailored to the categorical nature of the data. This article gives a systematic overview of penalty-based methods for categorical data developed so far and highlights some issues where further research is needed. We deal with categorical predictors as well as models for categorical response variables. The primary interest of this article is to give insight into basic properties of and differences between methods that are important with respect to statistical modelling in practice, without going into technical details or extensive discussion of asymptotic properties.