Watching the BiG artifacts: Exposing DeepFake videos via Bi-granularity artifacts. (March 2023)

• Record Type: Journal Article
• Authors: Chen, Han; Li, Yuezun; Lin, Dongdong; Li, Bin; Wu, Junqiang
• Is Part Of: Pattern recognition. Volume 135(2023)
• Publication Date: 2023-03
• Subjects: Multimedia forensics -- Deepfake detection -- Granularity artifacts -- Multi-task learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109179
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We expose DeepFake videos by detecting Bi-granularity artifacts, which are introduced by the common steps in DeepFake generation. We formulate DeepFake detection as a multi-task learning problem, where predicting Bi-granularity artifacts serves as auxiliary to further boost the detection. Our method shows favorable performance in within- and cross-dataset scenarios. Abstract: Recent years have witnessed significant advances in AI-based face manipulation techniques, known as DeepFakes, which has brought severe threats to society. Hence, an emerging and increasingly important research topic is how to detect DeepFake videos. In this paper, we propose a new DeepFake detection method based on Bi-granularity artifacts (BiG-Arts). We observe that the most of DeepFake video generation can commonly introduce bi-granularity artifacts: the intrinsic-granularity artifacts and extrinsic-granularity artifacts. Specifically, the intrinsic-granularity artifacts are caused by a common series of operations in model generation such as up-convolution or up-sampling, while the extrinsic-granularity artifacts are introduced by a common step in post-processing that blends the synthesized face to original video. To this end, we formulate DeepFake detection as multi-task learning problem, to simultaneously predict the intrinsic and extrinsic artifacts. Benefiting from the guidance of detecting Bi-granularity artifacts, our method is notably boosted in both within-datasets and cross-datasets scenarios. Extensive experiments are conducted on several DeepFake datasets, which corroborates the superiority of our method. Our method has been contributed as a part of the solution to achieve the Top-1 rank in DFGC competition (https://competitions.codalab.org/competitions/29583 ).