Integrating Illumination, Motion, and Shape Models for Robust Face Recognition in Video. (14th January 2007)

• Record Type: Journal Article
• Authors: Xu Xu, Yilei Yilei; Roy-Chowdhury Roy-Chowdhury, Amit Amit; Patel Patel, Keyur Keyur
• Other Names: Boulgouris Boulgouris N. N. Academic Editor.
• Is Part Of: EURASIP journal on advances in signal processing. Volume 2008(2008)
• Publication Date: 2007-01-14
• Subjects: Signal processing -- Periodicals; Traitement du signal; Signal processing; Periodicals; 621.3822
• DOI: 10.1155/2008/469698
• Languages: English
• ISSNs: 1687-6172

Abstract

Abstract : The use of video sequences for face recognition has been relatively less studied compared to image-based approaches. In this paper, we present an analysis-by-synthesis framework for face recognition from video sequences that is robust to large changes in facial pose and lighting conditions. This requires tracking the video sequence, as well as recognition algorithms that are able to integrate information over the entire video; we address both these problems. Our method is based on a recently obtained theoretical result that can integrate the effects of motion, lighting, and shape in generating an image using a perspective camera. This result can be used to estimate the pose and structure of the face and the illumination conditions for each frame in a video sequence in the presence of multiple point and extended light sources. We propose a new inverse compositional estimation approach for this purpose. We then synthesize images using the face model estimated from the training data corresponding to the conditions in the probe sequences. Similarity between the synthesized and the probe images is computed using suitable distance measurements. The method can handle situations where the pose and lighting conditions in the training and testing data are completely disjoint. We show detailed performance analysis results and recognition scores on a large video dataset.