A multimodal fake news detection model based on crossmodal attention residual and multichannel convolutional neural networks. Issue 1 (January 2021)

• Record Type: Journal Article
• Authors: Song, Chenguang; Ning, Nianwen; Zhang, Yunlei; Wu, Bin
• Is Part Of: Information processing & management. Volume 58:Issue 1(2021)
• Publication Date: 2021-01
• Subjects: Fake news detection -- Crossmodal attention -- Residual network -- Convolutional neural network; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2020.102437
• Languages: English
• ISSNs: 0306-4573

Abstract

Highlights: A multimodal fake news detection model based on crossmodal attention residual network and multichannel convolutional neural network is proposed. Our model can fuse the relevant information between different modalities while keeping the unique properties for each modality and alleviate the influence of noise information which may be generated by crossmodal fusion. The proposed method outperforms state-of-the-art methods and learns more discriminable feature representations. We contribute a large scale multimodal fake news dataset from Weibo platform and will make it available to the public. Abstract: In recent years, social media has increasingly become one of the popular ways for people to consume news. As proliferation of fake news on social media has the negative impacts on individuals and society, automatic fake news detection has been explored by different research communities for combating fake news. With the development of multimedia technology, there is a phenomenon that cannot be ignored is that more and more social media news contains information with different modalities, e.g., texts, pictures and videos. The multiple information modalities show more evidence of the happening of news events and present new opportunities to detect features in fake news. First, for multimodal fake news detection task, it is a challenge of keeping the unique properties for each modality while fusing the relevant information between different modalities. Second, for some news, the information fusion between different modalities may produce the noise information which affects model's performance. Unfortunately, existing methods fail to handle these challenges. To address these problems, we propose a multimodal fake news detection framework based on C rossmodal A ttention R esidual and M ultichannel convolutional neural N etworks (CARMN). The C rossmodal A ttention R esidual N etwork (CARN) can selectively extract the relevant information related to a target modality from another source modality while maintaining the unique information of the target modality. The M ultichannel C onvolutional neural N etwork (MCN) can mitigate the influence of noise information which may be generated by crossmodal fusion component by extracting textual feature representation from original and fused textual information simultaneously. We conduct extensive experiments on four real-world datasets and demonstrate that the proposed model outperforms the state-of-the-art methods and learns more discriminable feature representations.