PictureSensation – a mobile application to help the blind explore the visual world through touch and sound. (October 2016)

• Record Type: Journal Article
• Authors: Banf, Michael; Mikalay, Ruben; Watzke, Baris; Blanz, Volker
• Is Part Of: Journal of rehabilitation and assistive technologies engineering. Volume 3(2016)
• Publication Date: 2016-10
• Subjects: Visually impaired -- computer vision -- human computer interaction -- accessibility -- mobile computing; Rehabilitation technology -- Periodicals; 617.03
• DOI: 10.1177/2055668316674582
• Languages: English
• ISSNs: 2055-6683

Abstract

We present PictureSensation, a mobile application for the hapto-acoustic exploration of images. It is designed to allow for the visually impaired to gain direct perceptual access to images via an acoustic signal. PictureSensation introduces a swipe-gesture based, speech-guided, barrier free user interface to guarantee autonomous usage by a blind user. It implements a recently proposed exploration and audification principle, which harnesses exploration methods that the visually impaired are used to from everyday life. In brief, a user explores an image actively on a touch screen and receives auditory feedback about its content at his current finger position. PictureSensation provides an extensive tutorial and training mode, to allow for a blind user to become familiar with the use of the application itself as well as the principles of image content to sound transformations, without any assistance from a normal-sighted person. We show our application's potential to help visually impaired individuals explore, interpret and understand entire scenes, even on small smartphone screens. Providing more than just verbal scene descriptions, PictureSensation presents a valuable mobile tool to grant the blind access to the visual world through exploration, anywhere.