Holographic sign language avatar interpreter: A user interaction study in a mixed reality classroom. (9th June 2022)

• Record Type: Journal Article
• Authors: Yang, Fu‐Chia; Mousas, Christos; Adamo, Nicoletta
• Is Part Of: Computer animation and virtual worlds. Volume 33:Number 3/4(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2022-06-09
• Subjects: holographic avatar -- mixed reality -- sign language animation -- user interaction; Computer animation -- Periodicals; Visualization -- Periodicals; 006.6
• DOI: 10.1002/cav.2082
• Languages: English
• ISSNs: 1546-4261

Abstract

Abstract: We explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters. We recruited eight participants and conducted a usability test focused on avatar framing (full‐body vs. half‐body) and avatar manipulation (fixed position, scale, and orientation vs. user‐adjustable position, scale, and orientation) in the MR classroom. We used a mixed‐method approach to analyze quantitative and qualitative data through recordings, surveys, and interviews. The results show user preferences toward viewing holographic signing avatars in the MR environment and user acceptability toward such applications. Abstract : We explored user interactions with a holographic sign language interpreter in a mixed reality (MR) classroom for deaf and hard of hearing students. The developed MR application projects a holographic signing avatar that translates in real time the lecture while a speaking instructor is teaching. Our study explored user interaction with the MR system, intending to provide design guidelines for digital MR sign language interpreters.