Accurate 3D face reconstruction via prior constrained structure from motion. (August 2017)

• Record Type: Journal Article
• Authors: Hernandez, Matthias; Hassner, Tal; Choi, Jongmoo; Medioni, Gerard
• Is Part Of: Computers & graphics. Volume 66(2017)
• Page Start: 14
• Page End: 22
• Publication Date: 2017-08
• Subjects: 3D face reconstruction -- Face tracking -- Structure from motion -- 3DMM; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2017.05.008
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: We propose a novel 3D face modeling method designed for in-the-wild facial videos. Our method combines a statistical shape prior with Structure-from-motion techniques. We maximize photometric consistency over shape, expressions and camera poses. Our approach can provide with more accurate results than currently available methods. Our approach was quantitatively evaluated on a large publicly available datasets. Graphical abstract: Abstract: We propose a novel structure from motion (SfM) based method for reconstructing the 3D shapes of faces appearing in unconstrained videos. SfM techniques were studied since the early days of computer vision and their limitations are well known. In particular, they are unsuited for the low resolution scenes typical of videos of faces in the wild. To address this, we propose using a parametric face representation as a shape prior to constrain the estimated 3D face shape. Our proposed reconstruction method explores the space around the prior, modifying its 3D shape along with the estimated, per frame cameras and expressions in a manner designed to maximize photometric consistency across video frames yet produce face shapes. Our tests show that this process is both faster and provides superior 3D reconstruction accuracy when compared to existing alternatives.