Interactive spatial sonification for non-visual exploration of virtual maps. Issue 85 (January 2016)

• Record Type: Journal Article
• Authors: Geronazzo, Michele; Bedin, Alberto; Brayda, Luca; Campus, Claudio; Avanzini, Federico
• Is Part Of: International journal of human-computer studies. Issue 85(2016)
• Page Start: 4
• Page End: 15
• Publication Date: 2016-01
• Subjects: Multimodal interaction -- 3D audio -- Binaural sound -- Virtual maps -- Non-visual navigation -- Haptics -- Visual impairment -- Multisensory integration -- Haptic mouse; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2015.08.004
• Languages: English
• ISSNs: 1071-5819

Abstract

Abstract: This paper presents a multimodal interactive system for non-visual (auditory-haptic) exploration of virtual maps. The system is able to display haptically the height profile of a map, through a tactile mouse. Moreover, spatial auditory information is provided in the form of virtual anchor sounds located in specific points of the map, and delivered through headphones using customized Head-Related Transfer Functions (HRTFs). The validity of the proposed approach is investigated through two experiments on non-visual exploration of virtual maps. The first experiment has a preliminary nature and is aimed at assessing the effectiveness and the complementarity of auditory and haptic information in a goal reaching task. The second experiment investigates the potential of the system in providing subjects with spatial knowledge: specifically in helping with the construction of a cognitive map depicting simple geometrical objects. Results from both experiments show that the proposed concept, design, and implementation allow to effectively exploit the complementary natures of the "proximal" haptic modality and the "distal" auditory modality. Implications for orientation & mobility (O&M) protocols for visually impaired subjects are discussed. Abstract : Graphical abstract: Abstract : Highlights: We develop a multimodal interactive system for nonvisual exploration of virtual maps. Ecological and minimalistic principles guide customized binaural audio reproduction. Goal reaching and map reconstruction tasks assess contributions for each modality. The "proximal" haptic modality and the "distal" auditory modality are complementary. 3D auditory feedback improves performances with respect to 2D auditory feedback.