Spatiotemporal feature extraction for facial expression recognition. Issue 7 (1st July 2016)

• Record Type: Journal Article
• Authors: Kamarol, Siti Khairuni Amalina; Jaward, Mohamed Hisham; Parkkinen, Jussi; Parthiban, Rajendran
• Is Part Of: IET image processing. Volume 10:Issue 7(2016)
• Page Start: 534
• Page End: 541
• Publication Date: 2016-07-01
• Subjects: face recognition -- emotion recognition -- image sequences -- image classification -- image representation -- video signal processing -- support vector machines -- image texture; spatiotemporal feature extraction -- facial expression recognition -- appearance‐based facial feature extraction -- video sequences -- spatiotemporal texture map -- STTM -- spatial facial expression variation -- temporal facial expression variation -- low computational complexity -- Viola‐Jones face detector -- facial feature modelling -- 3D Harris corner function -- block‐based method -- feature representation -- support vector machine classifier -- posed expressions -- spontaneous microexpressions -- close‐to‐real‐world expressions; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2015.0519
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : A key issue regarding feature extraction is the capability of a technique to extract distinctive features to represent facial expressions while requiring a low computational complexity. In this study, the authors propose a novel approach for appearance‐based facial feature extraction to perform the task of facial expression recognition on video sequences. The proposed spatiotemporal texture map (STTM) is capable of capturing subtle spatial and temporal variations of facial expressions with low computational complexity. First, face is detected using Viola–Jones face detector and frames are cropped to remove unnecessary background. Facial features are then modelled with the proposed STTM, which uses the spatiotemporal information extracted from three‐dimensional Harris corner function. A block‐based method is adopted to extract the dynamic features and represent the features in the form of histograms. The features are then classified into classes of emotion by the support vector machine classifier. The experimental results demonstrate that the proposed approach shows superior performance compared with the state‐of‐the‐art approaches with an average recognition rate of 95.37, 98.56, and 84.52% on datasets containing posed expressions, spontaneous micro‐expressions, and close‐to‐real‐world expressions, respectively. They also show that the proposed algorithm requires low computational cost.