Image manipulation detection by multiple tampering traces and edge artifact enhancement. (January 2023)

• Record Type: Journal Article
• Authors: Lin, Xun; Wang, Shuai; Deng, Jiahao; Fu, Ying; Bai, Xiao; Chen, Xinlei; Qu, Xiaolei; Tang, Wenzhong
• Is Part Of: Pattern recognition. Volume 133(2023)
• Publication Date: 2023-01
• Subjects: Image manipulation detection -- Transformer -- Edge artifact enhancement -- Edge supervision; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109026
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract : highlights: Present an end-to-end EMT-Net for image manipulation detection. By fusing and enhancing tampering traces, it precisely predicts pixel-level results against multiple content-changing manipulation techniques; To detect heterologous and homogenous manipulation, transformer-based noise and CNN-based RGB encoding branches are developed. The two branches explore and fuse multiple tampering traces ( i.e., global and local noise, and visual artifact features) for better generalization; The proposed Edge decoding branch (EDB) reinforces tampering traces at different scales to distinguish boundary artifacts and natural object edges. Edge artifact enhancement (EAE) modules and edge supervision strategy in EDB extract subtle edge artifacts of manipulated regions despite applying post-processing methods; Experiments on six popular benchmarks indicate EMT-Net outperforming state-of-the-art approaches. EMT-Net is robust to images manipulated with various post-processing methods. Graphical abstract: Abstract: Image manipulation detection has attracted considerable attention owing to the increasing security risks posed by fake images. Previous studies have proven that tampering traces hidden in images are essential for detecting manipulated regions. However, existing methods have limitations in generalization and the ability to tackle post-processing methods. This paper presents a novel Network to learn and Enhance Multiple tampering Traces (EMT-Net), including noise distribution and visual artifacts. For better generalization, EMT-Net extracts global and local noise features from noise maps using transformers and captures local visual artifacts from original RGB images using convolutional neural networks. Moreover, we enhance fused tampering traces using the proposed edge artifacts enhancement modules and edge supervision strategy to discover subtle edge artifacts hidden in images. Thus, EMT-Net can prevent the risks of losing slight visual clues against well-designed post-processing methods. Experimental results indicate that the proposed method can detect manipulated regions and outperform state-of-the-art approaches under comprehensive quantitative metrics and visual qualities. In addition, EMT-Net shows robustness when various post-processing methods further manipulate images.