The Performance Index of Convolutional Neural Network-Based Classifiers in Class Imbalance Problem. (May 2023)

• Record Type: Journal Article
• Authors: Liu, Yanchen; Lai, King Wai Chiu
• Is Part Of: Pattern recognition. Volume 137(2023)
• Publication Date: 2023-05
• Subjects: Deep Learning -- Convolutional Neural Network -- Class Imbalance -- Class Balance Index -- Model Performance Index; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109284
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A class imbalance metric to assess classifier performance considering imbalance distribution impacts. Sensitive in comparison between classifier performances in different imbalance distribution statuses. An Algorithm for prediction on ideal classifier performance with a fair distribution (1:1) using limited size of samples. Abstract: Class imbalance is a common problem in many classification domains. This paper provides an evaluation index and one algorithm for this problem based on binary classification. The Model Performance Index ( MPI ) is proposed for assessing classifier performance as a new evaluation metric, considering class imbalance impacts. Based on MPI, we investigate algorithms to estimate ideal classifier performance with a fair distribution (1:1), referred to as the Ideal Model Performance Algorithm. Experimentally, compared with traditional metrics, MPI is more sensitive. Specifically, it can detect all types of changes in classifier performances, while others might remain at the same levels. Moreover, for the estimation of classifier performances, the algorithm reaches small differences between predictions and the values observed. Generally, for ideal performances, it achieved error rates of 0.060% - 1.3% for rare class in four experiments, showing a practical value on estimation and representation on the classifier performances.