Threshold optimization for F measure of macro-averaged precision and recall. (June 2020)

• Record Type: Journal Article
• Authors: Berger, Anna; Guda, Sergey
• Is Part Of: Pattern recognition. Volume 102(2020:Jun.)
• Publication Date: 2020-06
• Subjects: Macro-averaged F measure -- Multi-label classification -- Optimal threshold selection -- Fixed point method; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107250
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Coordinate-wise maximum for the analyzed measure may not be maximum in usual sense. The proposed fixed point method can localize all maximums of the measure. The method works the more precise, the bigger class count is. The approach is successfully applied to the real-world datasets. The difference from similar macro F measure is investigated. Abstract: There are two different approaches to macro-averaging F measure for multi-label classification. The first encloses averaging F measure over all classes, which makes it easy to optimize. The second, extensively investigated in this paper, comprises the F measure of macro precision and recall calculation. We examine and compare these two measures when applied to different multi-label datasets. To optimize the performance measure, we adopt a widely known and proven modular approach. Classifiers sort the instances in descending order, according to a real-valued score of belonging to a corresponding class. After that, thresholds are selected so as to optimize the performance measure. If the number of classes is sufficiently large and the second alternative of macro-averaging F measure is employed, then it becomes non-trivial to define the optimal number of instances to assign to each class. Cyclic optimization procedure is widely used for threshold optimization although it results in a maximum in a special coordinate-wise sense. For a micro averaged F measure, such a coordinate-wise optimum is a maximum in the conventional sense of this term but it is not true for the F measure of macro precision and recall, which is shown by a counterexample. We reduce the problem of selecting the optimal threshold for each class to the problem of obtaining a fixed point of a specifically introduced transformation of a unit square. The suggested algorithm lets us localize all possible coordinate-wise maximums and detect the optimal among them. The approach is applied to datasets from diverse application domains.