Performance Evaluation Indicator (PEI): A new paradigm to evaluate the competence of machine learning classifiers in predicting rockmass conditions. (January 2021)

• Record Type: Journal Article
• Authors: Zhu, Mengqi; Gutierrez, Marte; Zhu, Hehua; Ju, J. Woody; Sarna, Sharmin
• Is Part Of: Advanced engineering informatics. Volume 47(2021)
• Publication Date: 2021-01
• Subjects: Performance Evaluation Indicator (PEI) -- Rockmass classification -- TBM projects -- Machine learning classifiers -- Imbalanced database; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2020.101232
• Languages: English
• ISSNs: 1474-0346

Abstract

Abstract: To illustrate an unprejudiced comparison among machine learning classifiers established on proprietary databases, and to guarantee the validity and robustness of these classifiers, a Performance Evaluation Indicator (PEI) and the corresponding failure criterion are proposed in this study. Three types of machine learning classifiers, including the strictly binary classifier, the normal multiclass classifier and the misclassification cost-sensitive classifier, are trained on four datasets recorded from a water drainage TBM project. The results indicate that: (1) the PEI successfully compares the competence of classifiers under different scenarios by isolating the effects of different overlapping-degree of rockmass classes, and (2) the cost-sensitive algorithm is warranted to classify rockmasses when the ratio of inter-class classes is more than 8:1. The contributions of this research are to fill the gap in performance evaluations of a classifier for imbalanced training data, and to identify the best situation to apply this classifier.