Channel-Wise Spatiotemporal Aggregation Technology for Face Video Forensics. (29th August 2021)

• Record Type: Journal Article
• Authors: Lu, Yujiang; Liu, Yaju; Fei, Jianwei; Xia, Zhihua
• Other Names: Zhao Guoying Academic Editor.
• Is Part Of: Security and communication networks. Volume 2021(2021)
• Publication Date: 2021-08-29
• Subjects: Computer networks -- Security measures -- Periodicals; Computer security -- Periodicals; Cryptography -- Periodicals; 005.805
• DOI: 10.1155/2021/5524930
• Languages: English
• ISSNs: 1939-0114

Abstract

Abstract : Recent progress in deep learning, in particular the generative models, makes it easier to synthesize sophisticated forged faces in videos, leading to severe threats on social media about personal privacy and reputation. It is therefore highly necessary to develop forensics approaches to distinguish those forged videos from the authentic. Existing works are absorbed in exploring frame-level cues but insufficient in leveraging affluent temporal information. Although some approaches identify forgeries from the perspective of motion inconsistency, there is so far not a promising spatiotemporal feature fusion strategy. Towards this end, we propose the Channel-Wise Spatiotemporal Aggregation (CWSA) module to fuse deep features of continuous video frames without any recurrent units. Our approach starts by cropping the face region with some background remained, which transforms the learning objective from manipulations to the difference between pristine and manipulated pixels. A deep convolutional neural network (CNN) with skip connections that are conducive to the preservation of detection-helpful low-level features is then utilized to extract frame-level features. The CWSA module finally makes the real or fake decision by aggregating deep features of the frame sequence. Evaluation against a list of large facial video manipulation benchmarks has illustrated its effectiveness. On all three datasets, FaceForensics++, Celeb-DF, and DeepFake Detection Challenge Preview, the proposed approach outperforms the state-of-the-art methods with significant advantages.