A weighted information-gain measure for ordinal classification trees. (15th August 2020)

• Record Type: Journal Article
• Authors: Singer, Gonen; Anuar, Roee; Ben-Gal, Irad
• Is Part Of: Expert systems with applications. Volume 152(2020)
• Publication Date: 2020-08-15
• Subjects: Information-gain -- Decision trees -- Classification tree -- Weighted entropy -- C4.5 -- Ordinal classification; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2020.113375
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: We define an ordinal oriented information metric, based on weighted entropy. We propose an ordinal oriented decision-tree, using the new information metric. The new decision-tree method is effective for ordinal classification problems. The new tree outperforms C4.5 on most datasets with an ordinal target. The new tree outperforms Random Forest on several datasets with an ordinal target. Abstract: This paper proposes an ordinal decision-tree model, which applies a new weighted information-gain ratio (WIGR) measure for selecting the classifying attributes in the tree. The proposed measure utilizes a weighted entropy function that is defined proportionally to the value deviation of different classes and thus reflects the consequences of the magnitude of potential classification errors. The WIGR can be used to select the classifying attributes in decision trees in a manner that reduces risks. The proposed ordinal decision tree is found effective for classification problems in which the class variable exhibits some form of ordinal ordering, and where dependencies between the attributes and the class value can be non-monotonic. In a series of experiments based on publicly-known datasets, it is shown that the proposed ordinal decision tree outperforms its non-ordinal counterparts that utilize traditional entropy measures. The proposed model can be used as a part of an expert system for ordinal classification applications, such as health-state monitoring, portfolio investments classification and performance evaluation of service systems.