Lighting-aware face frontalization for unconstrained face recognition. (August 2017)

• Record Type: Journal Article
• Authors: Deng, Weihong; Hu, Jiani; Wu, Zhongjun; Guo, Jun
• Is Part Of: Pattern recognition. Volume 68(2017:Aug.)
• Page Start: 260
• Page End: 271
• Publication Date: 2017-08
• Subjects: Face frontalization -- Pose normalization -- Illumination normalization -- Unconstrained face recognition -- Labeled face in the wild learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.03.024
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Provide both lighting-recovered and lighting-normalized frontalized images. Basic frontalization with a generic 3D face model by the alignment of only five landmarks. Lighting recovered and normalized image filling by the symmetry of quotient image. LRFF method completes well with more sophisticated methods on the LFW benchmark. LNRR+LRA method outperforms the recent deep learning based methods on the MPIE database. Abstract: Face recognition under variable pose and lighting is still one of the most challenging problems, despite the great progress achieved in unconstrained face recognition in recent years. Pose variation is essentially a misalignment problem together with invisible region caused by self-occlusion. In this paper, we propose a lighting-aware face frontalization method that aims to generate both lighting-recovered and lighting-normalized frontalized images, based on only five fiducial landmarks. Basic frontalization is first performed by aligning a generic 3D face model into the input face and rendering it at frontal pose, with an accurate visible region estimation based on face borderline detection. Then we apply the illumination-invariant quotient image, estimated from the visible region, as a face symmetrical feature to fill the invisible region. Lighting-recovered face frontalization (LRFF) is conducted by rendering the estimated lighting on the invisible region. By adjusting the combination parameters, lighting-normalized face frontalization (LNFF) is performed by rendering the canonical lighting on the face. Although its simplicity, our LRFF method competes well with more sophisticated frontalization techniques, on the experiments of LFW database. Moreover, combined with our recently proposed LRA-based classifier, the LNFF based method outperforms the deep learning based methods by about 6% on the challenging experiment on Multiple PIE database under variable pose and lighting.