Portrait style transfer using deep convolutional neural networks and facial segmentation. (July 2020)

• Record Type: Journal Article
• Authors: Zhao, Huihuang; Zheng, Jinghua; Wang, Yaonan; Yuan, Xiaofang; Li, Yuhua
• Is Part Of: Computers & electrical engineering. Volume 85(2020)
• Publication Date: 2020-07
• Subjects: Deep convolutional neural networks -- Portrait -- Style transfer -- Facial segmentation; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2020.106655
• Languages: English
• ISSNs: 0045-7906

Abstract

Graphical abstract: Abstract: When standard neural style transfer approaches are used in portrait style transfer, they often inappropriately apply textures and colours in different regions of the style portraits to the content portraits, leading to unsatisfied transfer results. This paper presents a portrait style transfer method to transfer the style of one image to another. It first proposes a combined segmentation method for the portrait parts, which segments both the style portrait and the content portrait into masks of seven parts automatically, including background, face, eyes, nose, eyebrows, mouth and foreground. These masks are extracted to capture elements of the styles for objects in the style image and to preserve the structure in the content portrait. This paper then proposes an augmented deep Convolutional Neural Network (CNN) framework for portrait style transfer. The masks of seven parts are added into a trained deep convolutional neural network as feature maps in certain selected layers in the augmented deep CNN model. An improved loss function is proposed for the training of the portrait style transfer. Results on various images show that our method outperforms the state-of-the-art style transfer techniques.