A convex formulation for multiple ordinal output classification. (February 2019)

• Record Type: Journal Article
• Authors: Ma, Zhongchen; Chen, Songcan
• Is Part Of: Pattern recognition. Volume 86(2019:Feb.)
• Page Start: 73
• Page End: 84
• Publication Date: 2019-02
• Subjects: Multiple ordinal output classification -- Multiple discrete ordinal variables -- Ordinal regression -- Relationships -- Convex function; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2018.09.005
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: First study multiple ordinal output classification (MOOC) as a general machine learning task. Propose an effective formulation to jointly model the relationship among multiple ordinal dinal variables of MOOC and their discrete ordinal values. Exemplify a convex objective function by the formulation which allows us to learn the optimal model parameters and the relationships among output variables simultaneously. Apply the kernel trick to provide a nonlinear extension to enhance nonlinear ability of our model. Demonstrate that our method not only achieves effective classification performance but also reveals the structures among output variables. Abstract: Multiple ordinal output classification (MOOC) which specifically refers to learning an association between individual inputs (e.g. face images) and a set of discrete ordinal response/output variables (e.g. facial action units), is a special case of multi-output classification and also a relatively-understudied topic in machine learning. It is very challenging in how to jointly model the relationship among multiple output variables and their discrete ordinal values. In this paper, we propose an effective formulation to address the above challenging problems. Under this formulation, the objective function is convex and thus leads to a convex multiple ordinal output classification (ConMOOC). Specifically, we use a regularization formulation to model the relationships among multiple output variables and an effective threshold-based loss function to fit their ordinal values. To enhance ability of the model, we also apply the kernel trick to provide a nonlinear extension. For efficiency, we use an alternating iteration method to learn the optimal model parameters for each variable as well as the relationships between different variables. Experiments conducted on synthetic and real datasets demonstrate that ConMOOC not only achieves effective classification performance but also reveals the structures among output variables. To the best of our knowledge, MOOC as a general machine learning task is the first time to be studied.