Annotating and modeling empathy in spoken conversations. (July 2018)

• Record Type: Journal Article
• Authors: Alam, Firoj; Danieli, Morena; Riccardi, Giuseppe
• Is Part Of: Computer speech & language. Volume 50(2018)
• Page Start: 40
• Page End: 61
• Publication Date: 2018-07
• Subjects: Empathy -- Emotion -- Spoken conversation -- Behavior analysis -- Affective scene -- Affect -- Call center -- Human–Human conversation; Speech processing systems -- Periodicals; Automatic speech recognition -- Periodicals; Computers -- Periodicals; Linguistics -- Periodicals; Speech-Language Pathology -- Periodicals; Traitement automatique de la parole -- Périodiques; Reconnaissance automatique de la parole -- Périodiques; Automatic speech recognition; Speech processing systems; Electronic journals; Periodicals; 006.454
• DOI: 10.1016/j.csl.2017.12.003
• Languages: English
• ISSNs: 0885-2308

Abstract

Highlights: We address two related problems in automatic affective behavior analysis: the design of the annotation protocol and the automatic recognition of empathy from spoken conversations. We propose and evaluate an annotation scheme for empathy inspired by the modal model of emotions. We investigated features derived from the lexical and acoustic spaces. We evaluated automatic classification system on call center conversations, where it showed significantly better performance than the baseline. Abstract: Empathy, as defined in behavioral sciences, expresses the ability of human beings to recognize, understand and react to emotions, attitudes and beliefs of others. In this paper, we address two related problems in automatic affective behavior analysis: the design of the annotation protocol and the automatic recognition of empathy from human–human dyadic spoken conversations. We propose and evaluate an annotation scheme for empathy inspired by the modal model of emotions. The annotation scheme was evaluated on a corpus of real-life, dyadic spoken conversations. In the context of behavioral analysis, we designed an automatic segmentation and classification system for empathy. Given the different speech and language levels of representation where empathy may be communicated, we investigated features derived from the lexical and acoustic spaces. The feature development process was designed to support both the fusion and automatic selection of relevant features from a high dimensional space. The automatic classification system was evaluated on call center conversations where it showed significantly better performance than the baseline.