Face Space Representations in Deep Convolutional Neural Networks. Issue 9 (September 2018)

• Record Type: Journal Article
• Authors: O'Toole, Alice J.; Castillo, Carlos D.; Parde, Connor J.; Hill, Matthew Q.; Chellappa, Rama
• Is Part Of: Trends in cognitive sciences. Volume 22:Issue 9(2018)
• Page Start: 794
• Page End: 809
• Publication Date: 2018-09
• Subjects: convolutional neural networks -- deep learning -- face recognition -- visual cortex; Cognitive science -- Periodicals; Cognitive neuroscience -- Periodicals; 153.05
• DOI: 10.1016/j.tics.2018.06.006
• Languages: English
• ISSNs: 1364-6613

Abstract

Abstract : Inspired by the primate visual system, deep convolutional neural networks (DCNNs) have made impressive progress on the complex problem of recognizing faces across variations of viewpoint, illumination, expression, and appearance. This generalized face recognition is a hallmark of human recognition for familiar faces. Despite the computational advances, the visual nature of the face code that emerges in DCNNs is poorly understood. We review what is known about these codes, using the long-standing metaphor of a 'face space' to ground them in the broader context of previous-generation face recognition algorithms. We show that DCNN face representations are a fundamentally new class of visual representation that allows for, but does not assure, generalized face recognition. Highlights: Deep convolutional neural networks (DCNNs) are the first class of algorithm to achieve generalized face recognition across viewpoint, illumination, expression, and appearance. Face space illustrates the progress of automated face recognition. Image-based models do not generalize across images or identity. Active appearance models represent identity, but do not model image generalization. DCNNs create a unitary space that houses both facial identity and face images. Face representations in DCNNs are compact, with feature units that are not tuned to face or image properties (e.g., viewpoint) in any commonly understood way. DCNN face spaces retain highly detailed information about face images, in addition to face identities. Semantic interpretation of face representations in DCNN follows sparse trajectories in the space, rather than being interpretable by feature unit activation.