Credit scoring based on tree-enhanced gradient boosting decision trees. (1st March 2022)

• Record Type: Journal Article
• Authors: Liu, Wanan; Fan, Hong; Xia, Meng
• Is Part Of: Expert systems with applications. Volume 189(2022)
• Publication Date: 2022-03-01
• Subjects: Credit scoring -- Gradient boosting decision trees -- Feature augmentation -- Interpretability; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.116034
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Credit scoring is an important tool for banks and lending companies to realize credit risk exposure management and gain profits. GBDTs, a group of boosting-type ensemble algorithms, have shown promising improvement for credit scoring. However, GBDT improves the credit scoring performance by iteratively modifying only the fitting target for each base classifier and invariably works on the same features, which limits the diversity of individual classifiers in GBDT; Moreover, the performance-interpretability dilemma motivated a large number of works to focus on the pursuit of high-performance ensemble strategies, which leads to the lack of explorations on the interpretability of the credit scoring models. Based on the above limitations, two tree-based augmented GBDTs (AugBoost-RFS and AugBoost-RFU) are proposed in this work for credit scoring. In the proposed methods, a step-wise feature augmentation mechanism is introduced for GBDT to enrich the diversity of individual base classifiers; Tree-based embedding technologies simplify the process of feature augmentation and inherit interpretability of GBDT. Results on 4 large-scale credit scoring datasets show AugBoost-RFS/AugBoost-RFU outperforms GBDT; Besides, supervised tree-based step-wise feature augmentation for GBDT achieves comparable results to neural network-based step-wise feature augmentation while significantly improve the augmentation efficiency. Moreover, the intrinsic global interpreted results and decision rules of tree-enhanced GBDTs, as well as the marginal contributions of features that are visualized by TreeSHAP demonstrate AugBoost-RFS/AugBoost-RFU can be good candidates for interpretable credit scoring. Highlights: Two tree-based augmented GBDTs is proposed to improve the performance of credit scoring. Tree-based embedding improves the efficiency of augmentation for GBDT. The intrinsic interpretability of the proposed methods is studied. The marginal contribution of features is investigated by embedding TreeSHAP into tree-enhanced GBDT. Tree ensembled framework well balance the performance, efficiency, and interpretability of credit scoring.