A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system. (July 2015)

• Record Type: Journal Article
• Authors: Harley, Jason M.; Bouchet, François; Hussain, M. Sazzad; Azevedo, Roger; Calvo, Rafael
• Is Part Of: Computers in human behavior. Volume 48(2015)
• Page Start: 615
• Page End: 625
• Publication Date: 2015-07
• Subjects: Emotions -- Affect -- Computer-based learning environments -- Intelligent tutoring systems (ITS); Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2015.02.013
• Languages: English
• ISSNs: 0747-5632

Abstract

Highlights: Novel approach for measuring and synchronizing emotion data from three modalities. Modalities: facial expression, self-report, and electrodermal activity data. Examined if modalities identified the same emotions or complementary information. High level of agreement between the self-report and facial recognition modalities. Low level of agreement between electrodermal activation and others. Abstract: This paper presents the evaluation of the synchronization of three emotional measurement methods (automatic facial expression recognition, self-report, electrodermal activity) and their agreement regarding learners' emotions. Data were collected from 67 undergraduates enrolled at a North American University whom learned about a complex science topic while interacting with MetaTutor, a multi-agent computerized learning environment. Videos of learners' facial expressions captured with a webcam were analyzed using automatic facial recognition software (FaceReader 5.0). Learners' physiological arousal was recorded using Affectiva's Q-Sensor 2.0 electrodermal activity measurement bracelet. Learners' self-reported their experience of 19 different emotional states on five different occasions during the learning session, which were used as markers to synchronize data from FaceReader and Q-Sensor. We found a high agreement between the facial and self-report data (75.6%), but low levels of agreement between them and the Q-Sensor data, suggesting that a tightly coupled relationship does not always exist between emotional response components.