A vision for contextualized evaluation of remote collaboration supported by AR. (February 2022)

• Record Type: Journal Article
• Authors: Marques, Bernardo; Silva, Samuel; Teixeira, António; Dias, Paulo; Santos, Beatriz Sousa
• Is Part Of: Computers & graphics. Volume 102(2022)
• Page Start: 413
• Page End: 425
• Publication Date: 2022-02
• Subjects: Remote collaboration -- Augmented Reality -- Characterization collaborative process -- Guidelines -- Evaluation toolkit -- User study; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2021.10.009
• Languages: English
• ISSNs: 0097-8493

Abstract

Abstract: Remote collaboration using Augmented Reality (AR) has potential to support physically distributed team-members that need to achieve a common goal by increasing knowledge retention, improving understanding and awareness of the problem and its context. In this vein, the path to achieve usable, realistic and impactful solutions must entail an explicit understanding regarding how remote collaboration occurs through AR and how it may help contribute to a more effective work effort. Thus, characterization and evaluation of the collaborative process is paramount, but a particularly challenging endeavor, due to the multitude of aspects that define the collaboration effort. In this context, the work presented here contributes with a critical analysis, discussing current evaluation efforts, identifying limitations and opportunities. Then, we outline a conceptual framework to support researchers in conducting evaluations in a more structured manner. To instrument this vision, an evaluation toolkit is proposed to support contextual data collection and analysis in such scenarios and obtain an additional perspective on selected dimensions of collaboration. We illustrate the usefulness and versatility of the toolkit through a case study on remote maintenance, comparing two distinct methods: sharing of video and AR-based annotations. Last, we discuss the results obtained, showing the proposed vision allows to have an additional level of insights to better understand what happened, eliciting a more complete characterization of the work effort. Graphical abstract: Highlights: How evaluation of the collaborative process is conducted in AR-remote scenarios. Conceptual framework for conducting evaluations in a more structured manner. Creation of an evaluation toolkit for improved data collection and analysis. User study demonstrates the toolkit capabilities in a remote maintenance scenario. Toolkit elicits a more complete characterization of the collaborative process.