Exploring system wide trust prevalence and mitigation strategies with multiple autonomous agents. (June 2023)

• Record Type: Journal Article
• Authors: Walliser, James C.; de Visser, Ewart J.; Shaw, Tyler H.
• Is Part Of: Computers in human behavior. Volume 143(2023)
• Publication Date: 2023-06
• Subjects: Human-autonomy teaming -- Trust -- System-wide trust -- Autonomy -- Teaming -- Robotic swarms; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2023.107671
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: Studies have shown that people have a tendency to apply trust broadly across a system rather than apply specific trust in each component of a system. To date, this System–Wide Trust (SWT) effect has not been studied in the context of human-autonomy teaming. Moreover, attempts to mitigate this system-wide trust effect have not been studied extensively. Two studies were designed to address this research gap. In both studies, participants were teamed with four agents that utilized an autonomous target recognition (ATR) system to identify targets as enemy or friendly. The first study showed that when one of the autonomous agents was inaccurate and performance information was provided, participants were less accurate, more likely to verify the ATR's decision, spent more time verifying the ATR's decision, and rated the other systems as less trustworthy. The second study explored a mitigation strategy for the SWT effect employing a cognitive-behavioral intervention. Results showed that participants exposed to both correct system accuracy information and the training intervention about error attribution reduced the system-wide trust effect. These studies suggest that multi-agent systems should provide specialized cues and training to mitigate the system-wide trust effect when engaging with independent components within a system. Highlights: Calibrated trust in a multi-agent system involves trusting each agent individually. A single malfunctioning agent pulls down trust in other accurate agents. Providing system transparency information can reduce system-wide trust. Scenario-based training enhances trust calibration and monitoring behavior.