A practical approach to measuring user engagement with the refined user engagement scale (UES) and new UES short form. Issue 112 (April 2018) - O'Brien, Heather L.; Cairns, Paul; Hall, Mark
Highlights: The User Engagement Scale (UES) measures self-reported user engagement. We refined the factor structure of the UES using mirt for the R statistics program. We developed a new short form of the UES (UES-SF). The full UES and UES-SF were validated with a new data set. We offer guidance for adopting the UES and UES-SF in other studies. Abstract: User engagement (UE) and its measurement have been of increasing interest in human-computer interaction (HCI). The User Engagement Scale (UES) is one tool developed to measure UE, and has been used in a variety of digital domains. The original UES consisted of 31-items and purported to measure six dimensions of engagement: aesthetic appeal, focused attention, novelty, perceived usability, felt involvement, and endurability. A recent synthesis of the literature questioned the original six-factors. Further, the ways in which the UES has been implemented in studies suggests there may be a need for a briefer version of the questionnaire and more effective documentation to guide its use and analysis. This research investigated and verified a four-factor structure of the UES and proposed a Short Form (SF). We employed contemporary statistical tools that were unavailable during the UES' development to re-analyze the original data, consisting of 427 and 779 valid responses across two studies, and examined new data ( N =344) gathered as part of a three-year digital library project. In this paper we detail our analyses, present a revised long and short form (SF) version of the UES, and offer guidance for researchers interested in adopting the UES and UES-SF in their own studies.