Multiple camera in car audio–visual speech recognition using phonetic and visemic information. (October 2015)

• Record Type: Journal Article
• Authors: Biswas, Astik; Sahu, P.K.; Chandra, Mahesh
• Is Part Of: Computers & electrical engineering. Volume 47(2015)
• Page Start: 35
• Page End: 50
• Publication Date: 2015-10
• Subjects: AVICAR corpus -- AVSR -- Speech recognition -- AAM -- Phoneme -- Viseme; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2015.08.009
• Languages: English
• ISSNs: 0045-7906

Abstract

Graphical abstract: Abstract: This paper presents a phonetic and visemic information-based audio–visual speech recognizer (AVSR). Active appearance model (AAM) is used to extract the visual features as it finely represents the shape and appearance information extracted from jaw and lip region. Consideration of visual features along with traditional acoustic feature has been found to be promising in the complex auditory environment. However, most of the existing AVSR systems rarely faced the visual domain problems. In this work, a real world multiple camera corpus audio visual in car (AVICAR) is used for the speech recognition experiment. Texas Instruments and Massachusetts Institute of Technology (TIMIT) corpus sentence portion is used to study the performance of bimodal audio–visual speech recognizer. To consider "Mc-Guruk" effect, acoustic and visual models are trained according to phonetic and visemic information, respectively. Phonetic–visemic AVSR system shows significant improvement over phonetic AVSR system.