A proxy learning curve for the Bayes classifier. (April 2023)

• Record Type: Journal Article
• Authors: Salazar, Addisson; Vergara, Luis; Vidal, Enrique
• Is Part Of: Pattern recognition. Volume 136(2023)
• Publication Date: 2023-04
• Subjects: Classification -- Parameter learning -- Sample size -- Training set size -- Probability of error; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109240
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Finite training sets lead to classifier error rates above the Bayes error rate. The training set size required in a practical setting is a relevant information. Most approaches are purely experimental or consider just two classes. A theoretical expression for the relative excess over the Bayes error rate is provided. Abstract: In this paper, a theoretical learning curve is derived for the multi-class Bayes classifier. This curve fits general multivariate parametric models of the class-conditional probability density. The derivation uses a proxy approach based on analyzing the convergence of a statistic which is proportional to the posterior probability of the true class. By doing so, the curve depends only on the training set size and on the dimension of the feature vector; it does not depend on the model parameters. Essentially, the learning curve provides an estimate of the reduction in the excess of the probability of error that can be obtained by increasing the training set size. This makes it attractive in order to deal with the practical problems of defining appropriate training set sizes.