Speech emotion recognition with artificial intelligence for contact tracing in the COVID‐19 pandemic. Issue 1 (8th February 2023)

• Record Type: Journal Article
• Authors: Pucci, Francesco; Fedele, Pasquale; Dimitri, Giovanna Maria
• Is Part Of: Cognitive computation and systems. Volume 5:Issue 1(2023)
• Page Start: 71
• Page End: 85
• Publication Date: 2023-02-08
• Subjects: affective computing -- artificial intelligence -- machine learning; Cognitive science -- Periodicals; Artificial intelligence -- Periodicals; Neurosciences -- Periodicals; Computer science -- Periodicals; Neurosciences; Computer science; Cognitive science; Artificial intelligence; Periodicals; Electronic journals; 006.3
• DOI: 10.1049/ccs2.12076
• Languages: English
• ISSNs: 2517-7567

Abstract

Abstract: If understanding sentiments is already a difficult task in human‐human communication, this becomes extremely challenging when a human‐computer interaction happens, as for instance in chatbot conversations. In this work, a machine learning neural network‐based Speech Emotion Recognition system is presented to perform emotion detection in a chatbot virtual assistant whose task was to perform contact tracing during the COVID‐19 pandemic. The system was tested on a novel dataset of audio samples, provided by the company Blu Pantheon, which developed virtual agents capable of autonomously performing contacts tracing for individuals positive to COVID‐19. The dataset provided was unlabelled for the emotions associated to the conversations. Therefore, the work was structured using a sort of transfer learning strategy. First, the model is trained using the labelled and publicly available Italian‐language dataset EMOVO Corpus. The accuracy achieved in testing phase reached 92%. To the best of their knowledge, thiswork represents the first example in the context of chatbot speech emotion recognition for contact tracing, shedding lights towards the importance of the use of such techniques in virtual assistants and chatbot conversational contexts for psychological human status assessment. The code of this work was publicly released at: https://github.com/fp1acm8/SER .