Training neural network classifiers through Bayes risk minimization applying unidimensional Parzen windows. (May 2018)

• Record Type: Journal Article
• Authors: Lázaro, Marcelino; Hayes, Monson H.; Figueiras-Vidal, Aníbal R.
• Is Part Of: Pattern recognition. Volume 77(2018:May)
• Page Start: 204
• Page End: 215
• Publication Date: 2018-05
• Subjects: Bayes risk -- Parzen windows -- Binary classification; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.12.018
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: The main contribution of this manuscript is a new training algorithm for binary classification using neural networks. The training algorithm is based on the minimization of an estimate of the Bayes risk. Parzen windows method is used to estimate the conditional distributions necessary to compute the probabilities of error included in the Bayes risk. A new set of training algorithms emerge from this Bayes risk minimization formulation using Parzen windows. Some interesting relationships with classical training methods are discovered. Abstract: A new training algorithm for neural networks in binary classification problems is presented. It is based on the minimization of an estimate of the Bayes risk by using Parzen windows applied to the final one-dimensional nonlinear transformation of the samples to estimate the probability of classification error. This leads to a very general approach to error minimization and training, where the risk that is to be minimized is defined in terms of integrated one-dimensional Parzen windows, and the gradient descent algorithm used to minimize this risk is a function of the window that is used. By relaxing the constraints that are typically applied to Parzen windows when used for probability density function estimation, for example by allowing them to be non-symmetric or possibly infinite in duration, an entirely new set of training algorithms emerge. In particular, different Parzen windows lead to different cost functions, and some interesting relationships with classical training methods are discovered. Experiments with synthetic and real benchmark datasets show that with the appropriate choice of window, fitted to the specific problem, it is possible to improve the performance of neural network classifiers over those that are trained using classical methods.