Fusing Transformed Deep and Shallow features (FTDS) for image-based facial expression recognition. (15th October 2020)

• Record Type: Journal Article
• Authors: Bougourzi, F.; Dornaika, F.; Mokrani, K.; Taleb-Ahmed, A.; Ruichek, Y.
• Is Part Of: Expert systems with applications. Volume 156(2020)
• Publication Date: 2020-10-15
• Subjects: Facial expression -- PML Representation -- Deep features -- Hand-crafted features -- Cross-databases; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2020.113459
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: An improved face cropping scheme is proposed. Deep and Shallow features are fused for facial expression recognition. Transformed of Pyramid Multi-Level representation of three hand-crafted descriptors are used. Transformed of two deep features are used. Performance is studied using within-database and cross-databases validation schemes. Abstract: In this paper, we propose combining between the transformed hand-crafted and deep features using PCA to recognize the six-basic facial expressions from static images. To evaluate our approach, we use three popular databases (CK+, CASIA and MMI). We introduce the use of the Pyramid Multi Level (PML) face representation for facial expression recognition. The hand-crafted features are obtained with such representations. Initially, we determine the optimal level of the PML features of three hand-crafted descriptors (HOG, LPQ and BSIF) using CK+, CASIA and MMI databases. After the optimal level of the PML is found for each descriptor, we combine them together with the transformed final VGG-face layers (FC6 and FC7) in order to get a compact image descriptor. In within-database experiments, our approach achieved higher accuracy than the state-of-art methods on both the CK+ and CASIA databases, and competitive result on the MMI database. Likewise, our approach outperformed the static methods in all six experiments of cross-databases.