Deep facial emotion recognition in video using eigenframes. Issue 14 (14th October 2020)

• Record Type: Journal Article
• Authors: Hajarolasvadi, Noushin; Demirel, Hasan
• Is Part Of: IET image processing. Volume 14:Issue 14(2020)
• Page Start: 3536
• Page End: 3546
• Publication Date: 2020-10-14
• Subjects: principal component analysis -- emotion recognition -- computer vision -- face recognition -- video signal processing -- image representation -- image motion analysis; video‐based facial emotion recognition -- computer vision society -- single video -- eigenspace domain -- principal component analysis -- PCA -- single emotional video -- temporal motion variance -- facial expression dynamics -- temporal domain -- input eigenframes; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2019.1566
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : Recently, video‐based facial emotion recognition (FER) has been an attractive topic in the computer vision society. However, processing several hundreds of frames for a single video of a particular emotion is not efficient. In this study, the authors propose a novel approach to obtain a representative set of frames for a video in the eigenspace domain. Principal component analysis (PCA) is applied to a single emotional video extracting the most significant eigenframes representing the temporal motion variance embedded in the video. Given that faces are segmented and normalised, the variance captured by PCA is attributed to the facial expression dynamics. The variation in the temporal domain is mapped to the eigenspace reducing the redundancy. The proposed approach is used to extract the input eigenframes. Later, VGG‐16, ResNet50, and 2D and 3D CNN architectures called eigenFaceNet are trained on the RML, eNTERFACE′05, and AFEW 6.0 databases. The experimental results are superior to the state‐of‐the‐art by 8 and 4% for RML, eNTERFACE′05 databases, respectively. The performance achievement is also coupled with a reduction in the computational time.