Machine learning interpretability for a stress scenario generation in credit scoring based on counterfactuals. (15th September 2022)

• Record Type: Journal Article
• Authors: Bueff, Andreas C.; Cytryński, Mateusz; Calabrese, Raffaella; Jones, Matthew; Roberts, John; Moore, Jonathon; Brown, Iain
• Is Part Of: Expert systems with applications. Volume 202(2022)
• Publication Date: 2022-09-15
• Subjects: OR in banking -- Interpretable ML -- Credit scoring -- Stress scenario; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.117271
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: To boost the application of machine learning (ML) techniques for credit scoring models, the blackbox problem should be addressed. The primary aim of this paper is to propose a measure based on counterfactuals to evaluate the interpretability of a ML credit scoring technique. Counterfactuals assist with understanding the model with regard to the classification decision boundaries and evaluate model robustness. The second contribution is the development of a data perturbation technique to generate a stress scenario. We apply these two proposals to a dataset on UK unsecured personal loans to compare logistic regression and stochastic gradient boosting (SBG). We show that training a blackbox model (SGB) as conditioned on our data perturbation technique can provide insight into model performance under stressed scenarios. The empirical results show that our interpretability measure is able to capture the classification decision boundary, unlike AUC and the classification accuracy widely used in the banking sector. Highlights: Measure based on counterfactuals to evaluate global interpretability of models. Development of a data perturbation technique to generate a stress testing scenario. Generate different stressed scenarios from a UK unsecured personal loans dataset. Model performance is negatively impacted when loan default ratio is increased. Constraints give insight into feature values that correlate with misclassification.