Understanding the effects of real-time sentiment analysis and morale visualisation in backchannel systems: A case study. Issue 145 (January 2021)

• Record Type: Journal Article
• Authors: Wyeld, Theodor; Jiranantanagorn, Peerumporn; Shen, Haifeng; Liao, Kewen; Bednarz, Tomasz
• Is Part Of: International journal of human-computer studies. Issue 145(2021)
• Publication Date: 2021-01
• Subjects: Backchannel system -- Emotion -- Sentiment -- Morale -- Visualisation; Human-machine systems -- Periodicals; Systems engineering -- Periodicals; Human engineering -- Periodicals; Human engineering; Human-machine systems; Systems engineering; Periodicals; Electronic journals; 004.019
• DOI: 10.1016/j.ijhcs.2020.102524
• Languages: English
• ISSNs: 1071-5819

Abstract

Highlights: Design and evaluation of a backchannel system for connecting presenters with audience in realtime. After reviewing existing systems, the best components were included and new components added to improve overall performance and usability. Using a seven part principles-based evaluation system, analysis of the effectives of the tool possible as well as user attitude towards use of the tool. This was also analysed and confirms the tools effectiveness. A Markov chain was computed to confirm initial findings as well as predict possible future uptake of the tool. Some recommendations about future research directions and applicability of this tool in a wider context – other than education alone – is discussed. Abstract: When presenting to a large group of students, either in an amphitheatre or through an online platform, effectively connecting to the audience – understanding how well the audience is following the presentation and taking appropriate actions promptly if they experience difficulties – is a serious challenge. Backchannel systems are sometimes deployed to address this issue by allowing audience to give feedback to the presenter without interrupting the current discourse. However, these systems are not designed to immediately aggregate and present the audience's feedback to the presenter in a meaningful way that is easy to quickly digest. To fill this gap, we have explored a proof-of-concept method for analysing the emotions and sentiments from the audience's feedback in real time and displaying to the presenter a morale graph showing a trend of the audience's overall reaction over time. This allows a presenter to effectively connected to their audience in real time, knowing whether their presentation is going well and what issues their audience may have in common at any specific moment. We have further implemented this method in an educational context, using a prototype backchannel system, known as ClasSense, for a lecturer to effectively connect to their students. This paper presents the evaluation of this system, which shows that lecturers accept and prefer the morale graph based user interface developed over other backchannel user interfaces that display all posts in chronological order. Students also positively expressed their agreement that the system not only makes their feedback an important part of the class but also increases their interactions with the lecturers. This is further confirmed with a Markov chain predicting the probability that students' and lecturers' survey results lead to their overall positive sentiment towards the tool. The flexibility of the ClasSense system suggests it may also be suitable in contexts other than education.