The effect of an artificial agent's vocal expressiveness on immediacy and learning. (16th December 2021)

• Record Type: Journal Article
• Authors: Fountoukidou, Sofia; Matzat, Uwe; Ham, Jaap; Midden, Cees
• Is Part Of: Journal of computer assisted learning. Volume 38:Number 2(2022)
• Page Start: 500
• Page End: 512
• Publication Date: 2021-12-16
• Subjects: artificial agent -- learning -- modeling -- vocal expressiveness; Computer-assisted instruction -- Periodicals; 371.334
• DOI: 10.1111/jcal.12632
• Languages: English
• ISSNs: 0266-4909

Abstract

Abstract: Background: Though pedagogical artificial agents are expected to play a crucial role in the years to come, earlier studies provide inconsistent results regarding their effect on learning. This might be because their potential for exhibiting subtle nonverbal behaviours we know from human teachers has been untapped. What is more, there is little evidence of the processes underlying the effect of nonverbal behaviours of teachers (either human or artificial) on learning, so as to better guide their practical application. Objectives: The aims of the current research were threefold: firstly, to examine the effect of an artificial agent's vocal expressiveness on non‐verbal immediacy (teachers' ability to increase psychological closeness through nonverbal communication). Secondly, to test whether an artificial agent showing strong vocal expressiveness will enhance affective and cognitive learning (perceived and actual), as compared to an artificial agent that shows weak vocal expressiveness. Thirdly, to examine whether the underlying mechanisms of motivation and attention explain the effect of immediacy (and thereby also of vocal expressiveness) on the two learning outcomes. Methods: The study used a between‐participants design, with the participants being randomly assigned to one of the two experimental conditions: artificial modelling with strong expressiveness and artificial modelling with weak vocal expressiveness. Results and conclusions: Results showed that an artificial agent with strong vocal expressiveness increased affective and perceived cognitive learning. Partial support was found for actual cognitive learning. What is more, our findings revealed that vocal expressiveness is related to affective and perceived cognitive learning because it promotes nonverbal immediacy. Finally, results provided evidence of motivation as a mediator of the path from immediacy to affective learning. Major takeaways: The current findings verify the important role of nonverbal immediacy found in traditional educational settings. However, showing that these results also apply to artificial teachers is essential, given that the educational landscape is changing and reshaping by artificial intelligence. Thus, taking into consideration the role of vocal expressiveness in the development of artificial teachers or voice assistants like Alexa, Siri, and Google assistant, as a way to enhance immediacy and affective experience of learners is of imperative value, since they will be found more and more in our societies. Lay Description: What is currently known about the subject matter: There is a positive effect of nonverbal cues of human teachers on students' learning The artificial agent's voice is important for learning. What their paper adds to this: Provides strong evidence of the positive effect of vocal expressiveness of artificial agents on learning outcomes. Reveals that vocal expressiveness is related to learning outcomes because it enhances immediacy. Examines the proposed mechanisms of motivation and attention as mediators of the path from immediacy to affective and cognitive learning. Implications of study findings for practitioners: Taking into consideration vocal expressiveness of artificial agents during the design process is essential for increased learning. The combination of the vocal parameters of pitch and speech rate is effective for constructing vocal expressiveness. By showing the underlying mechanisms of artificial agents' nonverbal cues (i.e., vocal cues) on learning, it helps designers to make a better decisions during construction. The study's findings are also pertinent for human teachers in improving students' learning in traditional educational settings.