Mitigating knowledge imbalance in AI-advised decision-making through collaborative user involvement. Issue 172 (April 2023)

• Record Type: Journal Article
• Authors: Gomez, Catalina; Unberath, Mathias; Huang, Chien-Ming
• Is Part Of: International journal of human-computer studies. Issue 172(2023)
• Publication Date: 2023-04
• Subjects: Decision-making -- Human–AI interaction -- Knowledge imbalance -- Trust calibration; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2022.102977
• Languages: English
• ISSNs: 1071-5819

Abstract

Abstract: Integrating artificial intelligence (AI) systems into decision-making tasks attempts to assist people by augmenting or complementing their abilities and ultimately improve task performance. However, when considering recommendations from modern "black box" intelligent systems, users are confronted with the decision of accepting or overriding AI's recommendations. These decisions are even more challenging to make when there exists a significant knowledge imbalance between the users and the AI system—namely, when people lack necessary task knowledge and are therefore unable to accurately complete the task on their own. In this work, we aim to understand people's behavior in AI-assisted decision-making tasks when faced with the challenge of knowledge imbalance and explore whether involving users in an AI's prediction generation process makes them more willing to follow the AI's recommendations and enhances their perception of collaboration. Our empirical study reveals that the involvement of users in generating AI recommendations during a task with notable knowledge imbalance causes them to be more willing to agree with the AI's suggestions and to perceive the AI agent and their collaboration as a team more positively. Highlights: Agreement with the AI is promoted by involving users in the AI's process to generate a recommendation in a task with knowledge imbalance. Users exhibit more trusting behavior toward the AI agent when they collaborated more closely with the AI. User involvement promotes more positive perceptions of the AI agent and their collaboration as a team. Involvement is good for developing trust in the AI, but it could become overtrust.