A landmark-free approach for automatic, dense and robust correspondence of 3D faces. (January 2023)

• Record Type: Journal Article
• Authors: Fan, Zhenfeng; Hu, Xiyuan; Chen, Chen; Wang, Xiaolian; Peng, Silong
• Is Part Of: Pattern recognition. Volume 133(2023)
• Publication Date: 2023-01
• Subjects: 3D face -- Dense correspondence -- Non-rigid registration; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108971
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We construct an automatic algorithm using high-entropy points instead of landmarks, and take both global and local features of faces into consideration for accurate semantic correspondence. We consider topological correspondence and propose a mesh correction algorithm to filter out non-uniform local deformations, and this helps the construction of a compact 3D face model. We leverage a prior model with shape and normal statistics to handle hard samples with outliers, noises and expressions. This benefits robustness of the correspondence process with supervised domain knowledge of faces. Abstract: Global dense registration of 3D faces commonly prioritizes correspondences of facial landmarks which are fiducial points for the anatomical structures. However, it is not always easy to pre-annotate the landmarks accurately in raw scans of 3D faces. Contrary to the current state-of-the-art in dense 3D face correspondence, we propose a general framework without pre-annotated landmarks, which promotes its robustness and allows the meshes to deform in a uniform manner. The proposed framework includes two stages: first the correspondences are established using a template face; and then we select some well-reconstructed samples to build a prior model and leverage it into the correspondence process of other samples. In both stages, the dense registration is revisited in two perspectives: semantic and topological correspondence. In the latter stage, we further incorporate shape and normal statistics of 3D faces to regularize the correspondence process for more robust results. This provides a feasible way to handle data with noises and occlusions, as well as large deformation caused by facial expressions. Our basic idea is to gradually refine the correspondence of individual points in a way global-to-local. At the same time, we solve the local-to-global deformation based on the refined correspondences. The two processes are alternated, and aided by some confidence checks for each individual points. In the experiments, the proposed method is evaluated both qualitatively and quantitatively on three datasets including two publicly available ones: FRGC v2.0 and BU-3DFE datasets, demonstrating its effectiveness.