Rule extraction with guarantees from regression models. (June 2022)

• Record Type: Journal Article
• Authors: Johansson, Ulf; Sönströd, Cecilia; Löfström, Tuwe; Boström, Henrik
• Is Part Of: Pattern recognition. Volume 126(2022)
• Publication Date: 2022-06
• Subjects: Rule extraction -- Interpretability -- Conformal prediction -- Explainable AI -- Predictive regression; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108554
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Almost all studies about rule extraction investigate classification. In this paper, we study rule extraction from opaque predictive regression models. Today, all black-box rule extraction methods suffer from potentially low fidelity on test data. By utilizing conformal prediction in a novel way, the fidelity can be guaranteed, thus solving the main problem with black-box rule extraction. Another problem with rule extraction for regression is the choice of representation language; a standard regression tree with point predictions in the leaves is typically both too weak and convey very little information, while more complex alternatives like model trees are not truly comprehensible. We suggest a new representation language for the extracted models; i.e., standard regression trees, but augmented with valid and sharp prediction intervals in the leaves. In the extensive empirical investigation, the validity of the extracted models is demonstrated. In addition, it is shown how normalization can be used to provide individualized prediction intervals, thus providing highly informative extracted models. Abstract: Tools for understanding and explaining complex predictive models are critical for user acceptance and trust. One such tool is rule extraction, i.e., approximating opaque models with less powerful but interpretable models. Pedagogical (or black-box) rule extraction, where the interpretable model is induced using the original training instances, but with the predictions from the opaque model as targets, has many advantages compared to the decompositional (white-box) approach. Most importantly, pedagogical methods are agnostic to the kind of opaque model used, and any learning algorithm producing interpretable models can be employed for the learning step. The pedagogical approach has, however, one main problem, clearly limiting its utility. Specifically, while the extracted models are trained to mimic the opaque, there are absolutely no guarantees that this will transfer to novel data. This potentially low test set fidelity must be considered a severe drawback, in particular when the extracted models are used for explanation and analysis. In this paper, a novel approach, solving the problem with test set fidelity by utilizing the conformal prediction framework, is suggested for extracting interpretable regression models from opaque models. The extracted models are standard regression trees, but augmented with valid prediction intervals in the leaves. Depending on the exact setup, the use of conformal prediction guarantees that either the test set fidelity or the test set accuracy will be equal to a preset confidence level, in the long run. In the extensive empirical investigation, using 20 publicly available data sets, the validity of the extracted models is demonstrated. In addition, it is shown how normalization can be used to provide individualized prediction intervals, thus providing highly informative extracted models.