A two-part evaluation approach for measuring the usability and user experience of an Augmented Reality-based assistance system to support the temporal coordination of spatially dispersed teams. (August 2021)

• Record Type: Journal Article
• Authors: Thomaschewski, Lisa; Weyers, Benjamin; Kluge, Annette
• Is Part Of: Cognitive systems research. Volume 68(2021)
• Page Start: 1
• Page End: 17
• Publication Date: 2021-08
• Subjects: Usability -- User experience -- Augmented Reality -- Spatial dispersed teams -- Temporal coordination -- Artificial systems; Cognition -- Periodicals; Cognitive engineering (System design) -- Periodicals; Artificial intelligence -- Periodicals; 153.05
• DOI: 10.1016/j.cogsys.2020.12.001
• Languages: English
• ISSNs: 1389-0417

Abstract

Abstract: One key predictor of team performance and productivity is the teams' ability to coordinate its subtasks as precisely as possible over time. Thereby, the quality of the temporal coordination in teams is highly dependent on several cognitive team skills, like for instance the ability to build a precise and stable mental model of the situation (shared situation awareness) and of the team task process (task state awareness/ TSA). To support these cognitive skills in teams, various methods and trainings already exist. However, considering teams that work de-located, the prerequisites and therefore the requirements for team supportive methods change. This work presents the results of an empirical usability and user experience (UX) study of an interface for an Augmented Reality-based assistance system for spatially dispersed teams, called Ambient Awareness Tool (AAT). The AAT interface consists of graphical information about the teamwork process and aims at enhancing the TSA of the team, thereby supporting its temporal coordination. Within the framework of a participatory design process we conducted a two-part expert-user-study, comprising a laboratory experiment for the evaluation of the interface usability and a follow-up online survey to additionally investigate the UX. By means of the usability scores we first inferred three interface configuration clusters. A subsequent UX evaluation then allowed us to designate the configuration cluster with the highest usability and UX scores. As a result, we defined one interface configuration that will be investigated concerning its actual impact on the temporal coordination of spatially dispersed teams in a further study.