PolyHope: Two-level hope speech detection from tweets. (1st September 2023)

• Record Type: Journal Article
• Authors: Balouchzahi, Fazlourrahman; Sidorov, Grigori; Gelbukh, Alexander
• Is Part Of: Expert systems with applications. Volume 225(2023)
• Publication Date: 2023-09-01
• Subjects: Hope -- Wish -- Desire -- Expectation -- Machine learning -- Deep learning -- Transformers -- Natural Language Processing; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2023.120078
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Hope is characterized as openness of spirit towards the future, a desire, expectation, and wish for something to happen or to be true that remarkably affects human's state of mind, emotions, behaviors, and decisions. Hope is usually associated with concepts of desired expectations and possibility/probability concerning the future. Despite its importance, hope has rarely been studied as a social media analysis task. This paper presents a hope speech dataset that classifies each tweet first into "Hope" and "Not Hope", then into three fine-grained hope categories: "Generalized Hope", "Realistic Hope", and "Unrealistic Hope" (along with "Not Hope"). English tweets in the first half of 2022 were collected to build this dataset. Furthermore, we describe our annotation process and guidelines in detail and discuss the challenges of classifying hope and the limitations of the existing hope speech detection corpora. In addition, we reported several baselines based on different learning approaches, such as traditional machine learning, deep learning, and transformers, to benchmark our dataset. We evaluated our baselines using averaged-weighted and averaged-macro F1-scores. Observations show that a strict process for annotator selection and detailed annotation guidelines enhanced the dataset's quality. This strict annotation process yielded promising performance for simple machine learning classifiers with only uni-grams; however, binary and multiclass hope speech detection results reveal that contextual embedding models have higher performance in this dataset. Highlights: Hope is fine-grained and classified into different categories. Hope is directed towards either specific or general outcome. Simple n-grams feature are not sufficient for multiclass hope speech detection. Effectiveness of contextual embeddings and transformers on hope. Problems with the current understanding of hope.