"Magic mirror in my hand, what is the sentiment in the lens?": An action unit based approach for mining sentiments from multimedia contents. (April 2015)

• Record Type: Journal Article
• Authors: Casaburi, Luca; Colace, Francesco; De Santo, Massimo; Greco, Luca
• Is Part Of: Journal of visual languages & computing. Volume 27(2015)
• Page Start: 19
• Page End: 28
• Publication Date: 2015-04
• Subjects: Affective computing -- Ekman theory -- Emotional intelligence -- Human computer interaction; Visual programming languages (Computer science) -- Periodicals; Visual programming (Computer science) -- Periodicals; Programming languages (Electronic computers) -- Semantics -- Periodicals; Langages de programmation visuelle -- Périodiques; Programmation visuelle -- Périodiques; Langages de programmation -- Sémantique -- Périodiques; Programming languages (Electronic computers) -- Semantics; Visual programming (Computer science); Visual programming languages (Computer science); Periodicals; Electronic journals; 005
• DOI: 10.1016/j.jvlc.2015.01.001
• Languages: English
• ISSNs: 1045-926X

Abstract

Abstract: In psychology and philosophy, emotion is a subjective, conscious experience characterized primarily by psychophysiological expressions, biological reactions, and mental states. Emotion could be also considered as a "positive or negative experience" that is associated with a particular pattern of physiological activity. So, the extraction and recognition of emotions from multimedia contents is becoming one of the most challenging research topics in human–computer interaction. Facial expressions, posture, gestures, speech, emotive changes of physical parameters (e.g. body temperature, blush and changes in the tone of the voice) can reflect changes in the user׳s emotional state and all this kind of parameters can be detected and interpreted by a computer leading to the so-called "affective computing". In this paper an approach for the extraction of emotions from images and videos will be introduced. In particular, it involves the adoption of action units׳ extraction from facial expression according to the Ekman theory. The proposed approach has been tested on standard and real datasets with interesting and promising results. Highlights: Ekman׳s theory and AU combinations are effective in sentiment mining from multimedia contents. SURF Cascade algorithm shows good performance for face detection on video streams. AU combinations allow good performance on both standard and real datasets.