Automatic text‐to‐gesture rule generation for embodied conversational agents. (6th September 2020)

• Record Type: Journal Article
• Authors: Ali, Ghazanfar; Lee, Myungho; Hwang, Jae‐In
• Is Part Of: Computer animation and virtual worlds. Volume 31:Number 4/5(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-09-06
• Subjects: computer animation -- gesture generation -- rule‐based mapping -- social agents -- virtual agents; Computer animation -- Periodicals; Visualization -- Periodicals; 006.6
• DOI: 10.1002/cav.1944
• Languages: English
• ISSNs: 1546-4261

Abstract

Abstract: Interactions with embodied conversational agents can be enhanced using human‐like co‐speech gestures. Traditionally, rule‐based co‐speech gesture mapping has been utilized for this purpose. However, the creation of this mapping is laborious and often requires human experts. Moreover, human‐created mapping tends to be limited, therefore prone to generate repeated gestures. In this article, we present an approach to automate the generation of rule‐based co‐speech gesture mapping from publicly available large video data set without the intervention of human experts. At run‐time, word embedding is utilized for rule searching to get the semantic‐aware, meaningful, and accurate rule. The evaluation indicated that our method achieved comparable performance with the manual map generated by human experts, with a more variety of gestures activated. Moreover, synergy effects were observed in users' perception of generated co‐speech gestures when combined with the manual map. Abstract : In this article, we present an approach to automate the generation of rule‐based co‐speech gesture mapping from publicly available large video data set without the intervention of human experts. At run‐time, word embedding is utilized for rule searching to get the semantic‐aware, meaningful, and accurate rule. The evaluation indicated that our method achieved comparable performance with the manual map generated by human experts, with a more variety of gestures activated.