Deep generative image priors for semantic face manipulation. (July 2023)

• Record Type: Journal Article
• Authors: Hou, Xianxu; Shen, Linlin; Ming, Zhong; Qiu, Guoping
• Is Part Of: Pattern recognition. Volume 139(2023)
• Publication Date: 2023-07
• Subjects: GANs -- Face attribute prediction -- Semantic face manipulation; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109477
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel attribute-guided latent space optimization approach to control various facial attributes in the latent space of GANs. We propose a semantic-aware encoder for GAN inversion, which can not only faithfully reconstruct the input images, but also ensure that the inverted codes are semantically meaningful. We demonstrate the usefulness of synthesized face images for augmenting training data to improve weight initialization and accelerate the learning process for deep networks, leading to state-of-the-art performance for face attribute prediction. Abstract: Previous works on generative adversarial networks (GANs) mainly focus on how to synthesize high-fidelity images. In this paper, we present a framework to leverage the knowledge learned by GANs for semantic face manipulation. In particular, we propose to control the semantics of synthesized faces by adapting the latent codes with an attribute prediction model. Moreover, in order to achieve a more accurate estimation of different facial attributes, we propose to pretrain the attribute prediction model by inverting the synthesized face images back to the GAN latent space. As a result, our method explicitly considers the semantics encoded in the latent space of a pretrained GAN and is able to faithfully edit various attributes like eyeglasses, smiling, bald, age, mustache and gender for high-resolution face images. Extensive experiments show that our method has superior performance compared to state of the art for both face attribute prediction and semantic face manipulation.