A diagnostic framework for imbalanced classification in business process predictive monitoring. (1st December 2021)

• Record Type: Journal Article
• Authors: Kim, Jongchan; Comuzzi, Marco
• Is Part Of: Expert systems with applications. Volume 184(2021)
• Publication Date: 2021-12-01
• Subjects: Predictive monitoring -- Business process -- Resampling -- Class imbalance -- Event log; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115536
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Framework to diagnose poor recall on the minority class in process next activity prediction is proposed. In the input side, we provide an empirical comparison of different techniques to resample the input. In the output side, we propose a novel performance metric (adjusted recall) to detect low recall on the minority class. Experimental evaluation on 3 real life process event logs and framework usage guidelines are provided. Abstract: One of the use cases of business process predictive monitoring is predicting the next activity in a running case, which results in a multi-class classification problem. Approaches to this use case are usually evaluated considering average performance across all classes. This often masks poor performance on minority classes, particularly when classes to be predicted are imbalanced. This is the natural case in next activity prediction, where exceptions or optional activities occur, by design, less frequently than others. In this paper we propose a framework to diagnose poor predictive performance on the minority class in the next activity prediction use case that comprises two tools: an empirical comparison of different resampling techniques in the data preparation phase and a novel classification performance measure. The proposed performance measure aims at highlighting the poor recall on the minority class of a classifier, which is a particularly important performance in the context of next activity prediction, whereas the benchmark helps understanding which resampling technique would be the best at mitigating the poor recall. We also discuss how the two tools of the proposed framework can be combined from an AutoML perspective. The proposed framework has been evaluated on a set of publicly available event logs.