The false hope of current approaches to explainable artificial intelligence in health care. Issue 11 (November 2021)

• Record Type: Journal Article
• Authors: Ghassemi, Marzyeh; Oakden-Rayner, Luke; Beam, Andrew L
• Is Part Of: Lancet. Volume 3:Issue 11(2021)
• Page Start: e745
• Page End: e750
• Publication Date: 2021-11
• Subjects: Medical care -- Data processing -- Periodicals; Medical care -- Information technology -- Periodicals; Medical informatics -- Periodicals; 610.285
• DOI: 10.1016/S2589-7500(21)00208-9
• Languages: English
• ISSNs: 2589-7500

Abstract

Summary: The black-box nature of current artificial intelligence (AI) has caused some to question whether AI must be explainable to be used in high-stakes scenarios such as medicine. It has been argued that explainable AI will engender trust with the health-care workforce, provide transparency into the AI decision making process, and potentially mitigate various kinds of bias. In this Viewpoint, we argue that this argument represents a false hope for explainable AI and that current explainability methods are unlikely to achieve these goals for patient-level decision support. We provide an overview of current explainability techniques and highlight how various failure cases can cause problems for decision making for individual patients. In the absence of suitable explainability methods, we advocate for rigorous internal and external validation of AI models as a more direct means of achieving the goals often associated with explainability, and we caution against having explainability be a requirement for clinically deployed models.