Exploring the Influence of Focal Loss on Transformer Models for Imbalanced Maintenance Data in Industry 4.0. Issue 1 (2021)

• Record Type: Journal Article
• Authors: Usuga-Cadavid, Juan Pablo; Grabot, Bernard; Lamouri, Samir; Fortin, Arnaud
• Is Part Of: IFAC-PapersOnLine. Volume 54:Issue 1(2021)
• Page Start: 1023
• Page End: 1028
• Publication Date: 2021
• Subjects: Artificial Intelligence -- Natural Language Processing -- Predictive Maintenance -- Imbalanced Classification -- Deep Learning -- Transformers -- Transfer Learning; Automatic control -- Periodicals; 629.805
• DOI: 10.1016/j.ifacol.2021.08.121
• Languages: English
• ISSNs: 2405-8963

Abstract

Abstract: Harnessing data from historical maintenance databases may be challenging, as they tend to rely on text data provided by operators. Thus, they often include acronyms, jargon, typos, and other irregularities that complicate the automated analysis of such reports. Furthermore, maintenance datasets may present highly imbalanced distributions: some situations happen more often than others, which hinders the effective application of classic Machine Learning (ML) models. Hence, this paper explores the use of a recent Deep Learning (DL) architecture called Transformer, which has provided cutting-edge results in Natural Language Processing (NLP). To tackle the class imbalance, a loss function called Focal Loss (FL) is explored. Results suggests that when all the classes are equally important, the FL does not improve the classification performance. However, if the objective is to detect the minority class, the FL achieves the best performance, although by degrading the detection capacity for the majority class.