SynFace—Speech-Driven Facial Animation for Virtual Speech-Reading Support. (16th November 2009)

• Record Type: Journal Article
• Authors: Salvi, Giampiero; Beskow, Jonas; Al Moubayed, Samer; Granström, Björn
• Other Names: Bailly Gérard Academic Editor.
• Is Part Of: EURASIP journal on audio, speech, and music processing. Volume 2009(2009)
• Publication Date: 2009-11-16
• Subjects: Sound -- Recording and reproducing -- Digital techniques -- Periodicals; Computer sound processing -- Periodicals; Computer sound processing; Sound -- Recording and reproducing -- Digital techniques; Periodicals; Electronic journal; Electronic journals; 620.2
• DOI: 10.1155/2009/191940
• Languages: English
• ISSNs: 1687-4714

Abstract

Abstract : This paper describes SynFace, a supportive technology that aims at enhancing audio-based spoken communication in adverse acoustic conditions by providing the missing visual information in the form of an animated talking head. Firstly, we describe the system architecture, consisting of a 3D animated face model controlled from the speech input by a specifically optimised phonetic recogniser. Secondly, we report on speech intelligibility experiments with focus on multilinguality and robustness to audio quality. The system, already available for Swedish, English, and Flemish, was optimised for German and for Swedish wide-band speech quality available in TV, radio, and Internet communication. Lastly, the paper covers experiments with nonverbal motions driven from the speech signal. It is shown that turn-taking gestures can be used to affect the flow of human-human dialogues. We have focused specifically on two categories of cues that may be extracted from the acoustic signal: prominence/emphasis and interactional cues (turn-taking/back-channelling).