Distinguishing between fake news and satire with transformers. (January 2022)

• Record Type: Journal Article
• Authors: Low, Jwen Fai; Fung, Benjamin C.M.; Iqbal, Farkhund; Huang, Shih-Chia
• Is Part Of: Expert systems with applications. Volume 187(2022)
• Publication Date: 2022-01
• Subjects: Fake news -- Satire -- Sarcasm -- Deep learning -- Transformers -- BERT -- DistilBERT -- Classification; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115824
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Indiscriminate elimination of harmful fake news risks destroying satirical news, which can be benign or even beneficial, because both types of news share highly similar textual cues. In this work we applied a recent development in neural network architecture, transformers, to the task of separating satirical news from fake news. Transformers have hitherto not been applied to this specific problem. Our evaluation results on a publicly available and carefully curated dataset show that the performance from a classifier framework built around a DistilBERT architecture performed better than existing machine-learning approaches. Additional improvement over baseline DistilBERT was achieved through the use of non-standard tokenization schemes as well as varying the pre-training and text pre-processing strategies. The improvement over existing approaches stands at 0.0429 (5.2%) in F1 and 0.0522 (6.4%) in accuracy. Further evaluation on two additional datasets shows our framework's ability to generalize across datasets without diminished performance. Highlights: Transformers excel at fake news vs satire classification (FvS). Domain language adaptation via pre-training is crucial for FvS performance. Pre-training neural attention models even with tiny datasets can confer improvements. Greater granularity via multiple information aggregator tokens in text benefits FvS. Performance retained after dividing FvS into propaganda, clickbait, hoax, and satire.