Effects of Explainable Artificial Intelligence on trust and human behavior in a high-risk decision task. (February 2023)

• Record Type: Journal Article
• Authors: Leichtmann, Benedikt; Humer, Christina; Hinterreiter, Andreas; Streit, Marc; Mara, Martina
• Is Part Of: Computers in human behavior. Volume 139(2023)
• Publication Date: 2023-02
• Subjects: XAI -- AI literacy -- Domain-specific knowledge -- Mushroom identification -- Trust calibration -- Visual explanation; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2022.107539
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Understanding the recommendations of an artificial intelligence (AI) based assistant for decision-making is especially important in high-risk tasks, such as deciding whether a mushroom is edible or poisonous. To foster user understanding and appropriate trust in such systems, we assessed the effects of explainable artificial intelligence (XAI) methods and an educational intervention on AI-assisted decision-making behavior in a 2 × 2 between subjects online experiment with N = 410 participants. We developed a novel use case in which users go on a virtual mushroom hunt and are tasked with picking edible and leaving poisonous mushrooms. Users were provided with an AI-based app that showed classification results of mushroom images. To manipulate explainability, one subgroup additionally received attribution-based and example-based explanations of the AI's predictions; for the educational intervention one subgroup received additional information on how the AI worked. We found that the group that received explanations outperformed that which did not and showed better calibrated trust levels. Contrary to our expectations, we found that the educational intervention, domain-specific (i.e., mushroom) knowledge, and AI knowledge had no effect on performance. We discuss practical implications and introduce the mushroom-picking task as a promising use case for XAI research. Highlights: Explainable Artificial Intelligence (XAI) was studied in a novel high-risk mushroom hunting task. Visual explanations improved user performance and led to more appropriate trust levels. An educational intervention aiming to improve AI literacy had no effect on user performance. AI-assisted mushroom hunting is a promising use case for user studies on XAI.