Learning disentangling and fusing networks for face completion under structured occlusions. (March 2020)

• Record Type: Journal Article
• Authors: Li, Zhihang; Hu, Yibo; He, Ran; Sun, Zhenan
• Is Part Of: Pattern recognition. Volume 99(2020:Mar.)
• Publication Date: 2020-03
• Subjects: Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107073
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Face completion and corruption are treated as disentangling and fusing processes of clean faces and occlusions. Presenting the disentangling network and fusing network for face completion and corruption under structured occlusions. Without requiring the paired face images for training, we unify the face completion and corruption into a dual learning framework. Experiments are performed to show the effectiveness of DF-GAN for the face completion under structured occlusions. Abstract: Face completion aims to generate semantically new pixels for missing facial components. It is a challenging generative task due to large variations of face appearance. This paper studies generative face completion under structured occlusions. We treat the face completion and corruption as disentangling and fusing processes of clean faces and occlusions, and propose a jointly disentangling and fusing Generative Adversarial Network (DF-GAN). First, three domains are constructed, corresponding to the distributions of occluded faces, clean faces and structured occlusions. The disentangling and fusing processes are formulated as the transformations between the three domains. Then the disentangling and fusing networks are built to learn the transformations from unpaired data, where the encoder-decoder structure is adopted and allows DF-GAN to simulate structure occlusions by modifying the latent representations. Finally, the disentangling and fusing processes are unified into a dual learning framework along with an adversarial strategy. The proposed method is evaluated on Meshface verification problem. Experimental results on four Meshface databases demonstrate the effectiveness of our proposed method for the face completion under structured occlusions.