Beauty3DFaceNet: Deep geometry and texture fusion for 3D facial attractiveness prediction. (August 2021)

• Record Type: Journal Article
• Authors: Xiao, Qinjie; Wu, You; Wang, Dinghong; Yang, Yong-Liang; Jin, Xiaogang
• Is Part Of: Computers & graphics. Volume 98(2021)
• Page Start: 11
• Page End: 18
• Publication Date: 2021-08
• Subjects: 3D facial attractiveness prediction -- Deep learning -- Feature fusion -- Datasets; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2021.04.023
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: We propose the first deep learning network, called Beauty3DFaceNet, for 3D facial attractiveness assessment. It integrates facial geometry, facial texture, and facial prior and computes a more convincing 3D facial attractiveness score like human raters. We create a 3D facial attractiveness dataset ShadowFace3D, which contains sufficient information such as the point clouds, texture images, and texture mappings of 3D faces as well as their public aesthetic criteria. Specifically, it contains the original and the lifted face pair of the same person, designed by professional designers of plastic surgeons. It is the first 3D face database for attractiveness assessment. We present 3DFacePointNet++, a new network based on facial landmark priors to simulate the perceptual sensitivity of human eyes, which improves the performance of the Beauty3DFaceNet. Graphical abstract: Abstract: We present Beauty3DFaceNet, the first deep convolutional neural network to predict attractiveness on 3D faces with both geometry and texture information. The proposed network can learn discriminative and complementary 2D and 3D facial features, allowing accurate attractiveness prediction for 3D faces. The main component of our network is a fusion module that fuses geometric features and texture features. We further employ a novel sampling strategy for our network based on a prior of facial landmarks, which improves the performance of learning aesthetic features from a face point cloud. Comparing to previous work, our approach takes full advantage of 3D geometry and 2D texture and does not rely on handcrafted features based on highly accurate facial characteristics such as feature points. To facilitate 3D facial attractiveness research, we also construct the first 3D face dataset ShadowFace3D, which contains 6, 000 high-quality 3D faces with attractiveness labeled by human annotators. Extensive quantitative and qualitative evaluations show that Beauty3DFaceNet achieves a significant correlation with the average human ratings. This validates that a deep learning network can effectively learn and predict 3D facial attractiveness.