User experience with conversational agent: a systematic review of assessment methods. Issue 16 (10th December 2022)

• Record Type: Journal Article
• Authors: Tubin, Carla; Mazuco Rodriguez, João Pedro; de Marchi, Ana Carolina Bertoletti
• Is Part Of: Behaviour & information technology. Volume 41:Issue 16(2022)
• Page Start: 3519
• Page End: 3529
• Publication Date: 2022-12-10
• Subjects: Conversational agent -- user experience -- assessment; Electronic data processing -- Periodicals; Human engineering -- Periodicals; Information technology -- Periodicals; 303.4833
• DOI: 10.1080/0144929X.2021.2001047
• Languages: English
• ISSNs: 0144-929X

Abstract

ABSTRACT: Conversational agents are becoming popular for providing a more natural and realistic user experience. New studies have become significant to understand how to assess this experience, mainly because of the increase in applications of this nature. We systematically reviewed the literature to identify how the user experience is assessed when interacting with conversational agents. A total of 443 studies were identified in the ACM, IEEE, Springer, and Scopus databases. Of these, 27 studies met the eligibility criteria. Most studies used their own evaluation methods, without adopting questionnaires validated for UX evaluation. Few studies used assessment tools before participants interacted with agents, and only two carried out assessments before, during, and after use. The results of the assessments can be better if specific instruments for UX are adopted. Furthermore, it is necessary to assess the experience at different times and use combined methods, to understand aspects related to the participants' feelings and behaviours.