Representation-guided generative adversarial network for unpaired photo-to-caricature translation. (March 2021)

• Record Type: Journal Article
• Authors: Zheng, Ziqiang; Liu, Hongzhi; Yang, Fan; Zheng, Xingyu; Yu, Zhibin; Zhang, Shaoda
• Is Part Of: Computers & electrical engineering. Volume 90(2021)
• Publication Date: 2021-03
• Subjects: Photo-to-caricature -- Imitation -- Image-to-image translation -- Generative adversarial network; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2021.106999
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: Imitating the painting style of one caricature source is an interesting and important application. It requires to capture the caricature style from one reference image, and generate target caricature image with similar style representation based on one source photo. Recently, image-to-image translation is a proven and potential framework for photo-to-caricature task. However, it still suffers from three drawbacks: (1) annotating aligned photo-to-caricature pairs is expensive and time-consuming; (2) the photo-to-caricature requires to capture and exaggerate the high-level semantic representations; and (3) the multiple painting styles increase the translation difficulty. To tackle these issues, we propose an innovative representation-guided photo-to-caricature translation framework based on unpaired images. The representation-guided scheme is designed to transfer the selected caricature style. To improve image synthesis quality, we introduce one feature-pyramid adversarial network (FPAN ) to provide multiple feature-level constrains. The comprehensive experiments on various caricature datasets show excellent imitation capabilities of the proposed method. Graphical abstract: Highlights: We present an unpaired representation-guided framework for photo-to-caricature translation. We include a feature-pyramid adversarial training architecture to improve the translation quality. We create an additional information flow to improve the generator efficiency. The additional interaction offers clues to the generator and helps the generator distinguish the specific content and domain style latent codes more efficiently.