Detecting offensive speech in conversational code-mixed dialogue on social media: A contextual dataset and benchmark experiments. (1st April 2023)

• Record Type: Journal Article
• Authors: Madhu, Hiren; Satapara, Shrey; Modha, Sandip; Mandl, Thomas; Majumder, Prasenjit
• Is Part Of: Expert systems with applications. Volume 215(2023)
• Publication Date: 2023-04-01
• Subjects: Hate Speech -- Natural Language Processing -- Evaluation -- Conversational Analysis -- Benchmark -- Transformer; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.119342
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: The spread of Hate Speech on online platforms is a severe issue for societies and requires the identification of offensive content by platforms. Research has modeled Hate Speech recognition as a text classification problem that predicts the class of a message based on the text of the message only. However, context plays a huge role in communication. In particular, for short messages, the text of the preceding tweets can completely change the interpretation of a message within a discourse. This work extends previous efforts to classify Hate Speech by considering the current and previous tweets jointly. In particular, we introduce a clearly defined way of extracting context. We present the development of the first dataset for conversational-based Hate Speech classification with an approach for collecting context from long conversations for code-mixed Hindi (ICHCL dataset). Overall, our benchmark experiments show that the inclusion of context can improve classification performance over a baseline. Furthermore, we develop a novel processing pipeline for processing the context. The best-performing pipeline uses a fine-tuned SentBERT paired with an LSTM as a classifier. This pipeline achieves a macro F1 score of 0.892 on the ICHCL test dataset. Another KNN, SentBERT, and ABC weighting-based pipeline yields an F1 Macro of 0.807, which gives the best results among traditional classifiers. So even a KNN model gives better results with an optimized BERT than a vanilla BERT model. Highlights: Formulate Hate speech detection problem on social media conversational Dialogue. Develop a code-mixed English–Hindi dataset with a well-defined context. Proposes context representation scheme for conversational dialogue. Benchmarked proposed ICHCL dataset on state-of-art classifiers.