The effect of response complexity and media on user restatement with multimodal virtual assistants. Issue 119 (November 2018)

• Record Type: Journal Article
• Authors: Freeman, C.; Beaver, I.
• Is Part Of: International journal of human-computer studies. Issue 119(2018)
• Page Start: 12
• Page End: 27
• Publication Date: 2018-11
• Subjects: Intelligent virtual agents -- Multimodal automated assistants -- Multimodal conversation design; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2018.06.002
• Languages: English
• ISSNs: 1071-5819

Abstract

Highlights: We determine media and linguistic complexity features that reduce user restatements. Including all media forms may be unhelpful if linguistic complexity is high. Web content alone may increase the number of non-clarifying user restatements. Users like responses with shorter sentences and clauses regardless of overall length. Related topic links help users clarify their tasks when user input is ambiguous. Abstract: We investigate the occurrence of user restatement when there is no apparent error in Intelligent Virtual Assistant (IVA) understanding in a multimodal customer service environment. Analysis was performed on 2998 sessions with a live customer service IVA deployed on a major airline company website and mobile application. Restatements of a user's previous turn in a conversation are separated into two classes by evidence of dissatisfaction in the IVA response. We consider combinations of response media and linguistic complexity features and determine through detailed statistical analysis which combinations of features should be minimized to improve user comprehension. A discussion of these findings follows demonstrating how the presentation of response in addition to the formulation of response text in a multimodal environment can have an effect on user understanding. Through this analysis, 1 we derive guidelines when crafting responses for designers of multimodal IVAs.