Trust in an AI versus a Human teammate: The effects of teammate identity and performance on Human-AI cooperation. (February 2023)

• Record Type: Journal Article
• Authors: Zhang, Guanglu; Chong, Leah; Kotovsky, Kenneth; Cagan, Jonathan
• Is Part Of: Computers in human behavior. Volume 139(2023)
• Publication Date: 2023-02
• Subjects: Artificial intelligence -- Trust -- Deception -- Anthropomorphism -- Human-computer interaction -- Decision-making; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2022.107536
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Recent advances in artificial intelligence (AI) enable researchers to create more powerful AI agents that are becoming competent teammates for humans. However, human distrust of AI is a critical factor that may impede human-AI cooperation. Although AI agents have been endowed with anthropomorphic traits, such as a human-like appearance, in prior studies to improve human trust in AI, it is still an open question whether humans have more trust in an AI teammate and achieve better human-AI joint performance if they are deceived about the identity of their AI teammate as another human. This research assesses the effects of teammate identity ("human" vs. AI) and teammate performance (low-performing vs. high-performing AI) on human-AI cooperation through a human subjects study. The results of this study show that humans behaviorally trust the AI more than another human by accepting their AI teammate's decisions more often. In addition, teammate performance has a significant effect on human-AI joint performance in the study, while teammate identity does not. These results caution against deceiving humans about the identity of AI in future applications involving human-AI cooperation. Highlights: Humans accept their AI teammate's decision less often when they are deceived about the identity of the AI as another human. Teammate performance has a significant effect on human-AI joint performance while teammate identity does not. Humans perceive higher temporal demand when they work with the "human" teammate than the AI teammate. Humans report that the low-performing "human" teammate is more competent and helpful than the low-performing AI teammate. Human individual expertise in the task has a considerable influence on human-AI cooperation.