Multimodal emotion recognition based on peak frame selection from video. Issue 5 (July 2016)

• Record Type: Journal Article
• Authors: Zhalehpour, Sara; Akhtar, Zahid; Eroglu Erdem, Cigdem
• Is Part Of: Signal, image and video processing. Volume 10:Issue 5(2016)
• Page Start: 827
• Page End: 834
• Publication Date: 2016-07
• Subjects: Affective computing -- Facial expression recognition -- Apex frame -- Audio-visual emotion recognition; Signal processing -- Digital techniques -- Periodicals; Image processing -- Digital techniques -- Periodicals; Digital video -- Periodicals; 621.3822
• DOI: 10.1007/s11760-015-0822-0
• Languages: English
• ISSNs: 1863-1703

Abstract

Abstract We present a fully automatic multimodal emotion recognition system based on three novel peak frame selection approaches using the video channel. Selection of peak frames (i.e., apex frames) is an important preprocessing step for facial expression recognition as they contain the most relevant information for classification. Two of the three proposed peak frame selection methods (i.e., MAXDIST and DEND-CLUSTER) do not employ any training or prior learning. The third method proposed for peak frame selection (i.e., EIFS) is based on measuring the "distance" of the expressive face from the subspace of neutral facial expression, which requires a prior learning step to model the subspace of neutral face shapes. The audio and video modalities are fused at the decision level. The subject-independent audio-visual emotion recognition system has shown promising results on two databases in two different languages (eNTERFACE and BAUM-1a).