Audio cues enhance mirroring of arm motion when visual cues are scarce. Issue 154 (31st May 2019)

• Record Type: Journal Article
• Authors: Lee, Edward D.; Esposito, Edward; Cohen, Itai
• Is Part Of: Journal of the Royal Society interface. Volume 16:Issue 154(2019)
• Publication Date: 2019-05-31
• Subjects: virtual reality -- coordination -- motion capture -- mirroring -- statistical learning; Physical sciences -- Research -- Periodicals; Life sciences -- Research -- Periodicals; Interdisciplinary research -- Periodicals; 570.5
• DOI: 10.1098/rsif.2018.0903
• Languages: English
• ISSNs: 1742-5689

Abstract

Abstract : Swing in a crew boat, a good jazz riff, a fluid conversation: these tasks require extracting sensory information about how others flow in order to mimic and respond. To determine what factors influence coordination, we build an environment to manipulate incoming sensory information by combining virtual reality and motion capture. We study how people mirror the motion of a human avatar's arm as we occlude the avatar. We efficiently map the transition from successful mirroring to failure using Gaussian process regression. Then, we determine the change in behaviour when we introduce audio cues with a frequency proportional to the speed of the avatar's hand or train individuals with a practice session. Remarkably, audio cues extend the range of successful mirroring to regimes where visual information is sparse. Such cues could facilitate joint coordination when navigating visually occluded environments, improve reaction speed in human–computer interfaces or measure altered physiological states and disease.