Visual diagnostics of an explainer model: Tools for the assessment of LIME explanations. (19th February 2021)

• Record Type: Journal Article
• Authors: Goode, Katherine; Hofmann, Heike
• Is Part Of: Statistical analysis and data mining. Volume 14:Number 2(2021)
• Page Start: 185
• Page End: 200
• Publication Date: 2021-02-19
• Subjects: black‐box models -- data science -- explainable machine learning -- interpretability -- statistical graphics; Data mining -- Statistical methods -- Periodicals; 006.312
• DOI: 10.1002/sam.11500
• Languages: English
• ISSNs: 1932-1864

Abstract

Abstract: The importance of providing explanations for predictions made by black‐box models has led to the development of explainer model methods such as LIME (local interpretable model‐agnostic explanations). LIME uses a surrogate model to explain the relationship between predictor variables and predictions from a black‐box model in a local region around a prediction of interest. However, the quality of the resulting explanations relies on how well the explainer model captures the black‐box model in a specified local region. Here we introduce three visual diagnostics to assess the quality of LIME explanations: (1) explanation scatterplots, (2) assessment metric plots, and (3) feature heatmaps. We apply the visual diagnostics to a forensics bullet matching dataset to show examples where LIME explanations depend on the tuning parameter values and the explainer model oversimplifies the black‐box model. Our examples raise concerns about claims made of LIME that are similar to other criticisms in the literature.