Robust, Real-Time 3D Face Tracking from a Monocular View. (14th October 2010)

• Record Type: Journal Article
• Authors: Liao Liao, Wei-Kai Wei-Kai; Fidaleo Fidaleo, Douglas Douglas; Medioni Medioni, Gerard Gerard
• Other Names: Eisert Eisert Peter Peter Academic Editor.
• Is Part Of: EURASIP journal on image and video processing. Volume 2010(2010)
• Publication Date: 2010-10-14
• Subjects: Image processing -- Digital techniques -- Periodicals; Digital video -- Periodicals; Traitement d'images; Vidéo numérique; Digital video; Image processing -- Digital techniques; Periodicals; Electronic journal; Electronic journals; 621.367
• DOI: 10.1155/2010/183605
• Languages: English
• ISSNs: 1687-5176

Abstract

Abstract : This paper addresses the problem of 3D face tracking from a monocular view. Dominant tracking algorithms in current literature can be classified as intensity-based or feature-based methods. Intensity-based methods track 3D faces based on the brightness constraint, assuming constant intensity of the face across adjacent frames. Feature-based trackers use local 2D features to determine sparse pairs of corresponding points between two frames and estimate 3D pose from these correspondences. We argue that using either approach alone neglects valuable visual information used in the other method. We therefore propose a novel hybrid tracking approach that integrates multiple visual cues. The hybrid tracker uses a nonlinear optimization framework to incorporate both feature correspondence and brightness constraints, and achieves reliable 3D face tracking in real-time. We conduct a series of experiments to analyze our approach and compare its performance with other state-of-the-art trackers. The experiments consist of synthetic sequences with simulated environmental factors and real-world sequences with estimated ground truth. Results show that the hybrid tracker is superior in both accuracy and robustness, particularly when dealing with challenging conditions such as occlusion and extreme lighting. We close with a description of a real-world human-computer interaction application based on our hybrid tracker.