The description-experience gap in the effect of warning reliability on user trust and performance in a phishing-detection context. Issue 119 (November 2018)

• Record Type: Journal Article
• Authors: Chen, Jing; Mishler, Scott; Hu, Bin; Li, Ninghui; Proctor, Robert W.
• Is Part Of: International journal of human-computer studies. Issue 119(2018)
• Page Start: 35
• Page End: 47
• Publication Date: 2018-11
• Subjects: Description-experience gap -- Trust -- Human-automation interaction -- Phishing detection -- Feedback; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2018.05.010
• Languages: English
• ISSNs: 1071-5819

Abstract

Highlights: Reliability influenced user performance and trust in a phishing detection system. This influence depended on task difficulty. Feedback (experience) increased objective and subjective trust calibration. Providing description of system reliability increased mainly subjective trust. Abstract: How the human user trusts and interacts with an automation system is influenced by how well the system capabilities are conveyed to the user. When interacting with the automation system, the user can obtain the system reliability information through an explicit description of the reliability or through experiencing the system over time. The term description-experience gap illustrates the difference between description-based and experience-based human decisions. In the current study, we investigated how this description-experience gap applies to human-automation interaction with a phishing-detection task in the cyber domain. In two experiments, participants' performance in detecting phishing emails and their trust in the phishing detection system were measured when system reliability, description, and experience (i.e., feedback) were varied systematically in easy and difficult phishing detection tasks. The results suggested that system reliability had a profound influence on human performance in the system, but the benefits of having a more reliable system may depend on task difficulty. Also, providing feedback increased trust calibration in terms of both objective and subjective trust measures, yet providing description of system reliability increased only subjective trust. This result pattern not only shows the gap in the effects of feedback and description, but it also extends the description-experience gap concept from rare events to common events.