PRRNet: Pixel-Region relation network for face forgery detection. (August 2021)

• Record Type: Journal Article
• Authors: Shang, Zhihua; Xie, Hongtao; Zha, Zhengjun; Yu, Lingyun; Li, Yan; Zhang, Yongdong
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: Face forgery detection -- Forgery localization -- Inconsistency detection -- Relation learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107950
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel Pixel-Region Relation Network is proposed to exploit pixelwise and region-wise relations for face forgery detection. A pixel-wise relation module is proposed to represent the relation between every two pixels in feature map to enhance the discriminant ability of local features. A region-wise relation module is proposed to detect the inconsistency between regions by fusing multiple metrics. We achieve new state-of-the-art results on three face forgery detection datasets. Abstract: As advanced facial manipulation technologies develop rapidly, one can easily modify an image by changing the identity or the facial expression of the target person, which threatens social security. To address this problem, face forgery detection becomes an important and challenging task. In this paper, we propose a novel network, called Pixel-Region Relation Network (PRRNet), to capture pixel-wise and region-wise relations respectively for face forgery detection. The main motivation is that a facial manipulated image is composed of two parts from different sources, and the inconsistencies between the two parts is a significant kind of evidence for manipulation detection. Specifically, PRRNet contains two serial relation modules, i.e. the Pixel-Wise Relation (PR) module and the Region-Wise Relation (RR) module. For each pixel in the feature map, the PR module captures its similarities with other pixels to exploit the local relations information. Then, the PR module employs a spatial attention mechanism to represent the manipulated region and the original region separately. With the representations of the two regions, the RR module compares them with multiple metrics to measure the inconsistency between these two regions. In particular, the final predictions are obtained totally based on whether the inconsistencies exist. PRRNet achieves the state-of-the-art detection performance on three recent proposed face forgery detection datasets. Besides, our PRRNet shows the robustness when trained and tested on different image qualities.