A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. Issue 112 (April 2018)

• Record Type: Journal Article
• Authors: O'Brien, Heather L.; Cairns, Paul; Hall, Mark
• Is Part Of: International journal of human-computer studies. Issue 112(2018)
• Page Start: 28
• Page End: 39
• Publication Date: 2018-04
• Subjects: User engagement -- Questionnaires -- Measurement -- Reliability -- Validity; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2018.01.004
• Languages: English
• ISSNs: 1071-5819

Abstract

Highlights: The User Engagement Scale (UES) measures self-reported user engagement. We refined the factor structure of the UES using mirt for the R statistics program. We developed a new short form of the UES (UES-SF). The full UES and UES-SF were validated with a new data set. We offer guidance for adopting the UES and UES-SF in other studies. Abstract: User engagement (UE) and its measurement have been of increasing interest in human-computer interaction (HCI). The User Engagement Scale (UES) is one tool developed to measure UE, and has been used in a variety of digital domains. The original UES consisted of 31-items and purported to measure six dimensions of engagement: aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability. A recent synthesis of the literature questioned the original six-factors. Further, the ways in which the UES has been implemented in studies suggests there may be a need for a briefer version of the questionnaire and more effective documentation to guide its use and analysis. This research investigated and verified a four-factor structure of the UES and proposed a Short Form (SF). We employed contemporary statistical tools that were unavailable during the UES' development to re-analyze the original data, consisting of 427 and 779 valid responses across two studies, and examined new data ( N =344) gathered as part of a three-year digital library project. In this paper we detail our analyses, present a revised long and short form (SF) version of the UES, and offer guidance for researchers interested in adopting the UES and UES-SF in their own studies.