Analysis of sentiment in tweets addressed to a single domain-specific Twitter account: Comparison of model performance and explainability of predictions. (30th December 2021)

• Record Type: Journal Article
• Authors: Fiok, Krzysztof; Karwowski, Waldemar; Gutierrez, Edgar; Wilamowski, Maciej
• Is Part Of: Expert systems with applications. Volume 186(2021)
• Publication Date: 2021-12-30
• Subjects: Natural language processing -- Deep learning -- Sentiment analysis -- Machine learning -- Explainability -- Twitter; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115771
• Languages: English
• ISSNs: 0957-4174

Abstract

Highlights: Comparison of selected popular and recent natural language processing methods. Use of explainable Artificial Intelligence tools in Twitter sentiment analysis. Analysis of sentiment in tweets addressed to a single Twitter account. Performance of selected transformer models on the SemEval-2017 data set. Abstract: Many institutions and companies find it valuable to know how people feel about their ventures; hence, scientific research in sentiment analysis has been intensely developed over time. Automated sentiment analysis can be considered as a machine learning (ML) prediction task, with classes representing human affective states. Due to the rapid development of ML and deep learning (DL), improvements in automatic sentiment analysis performance are achieved almost every year. Since 2013, Semantic Evaluation (SemEval) has hosted a worldwide community-acknowledged competition that allows for comparisons of recent innovations. The sentiment analysis tasks focus on assessing sentiment in Twitter posts authored by various publishers and addressing multiple subjects. Our study aimed to compare selected popular and recent natural language processing methods using a new data set of Twitter posts sent to a single Twitter account. For improved comparability of our experiments with SemEval, we adopted their metrics and also deployed our models on data published for SemEval-2017. In addition, we investigated if an unsupervised ML technique applied for the detection of topics in tweets can be leveraged to improve the predictive performance of a selected transformer model. We also demonstrated how a recent explainable artificial intelligence technique can be used in Twitter sentiment analysis to gain a deeper understanding of the models' predictions. Our results show that the most recent DL language modeling approach provides the highest quality; however, this quality comes at reduced model transparency.