Model for predicting perception of facial action unit activation using virtual humans. (November 2021)

• Record Type: Journal Article
• Authors: McDonnell, Rachel; Zibrek, Katja; Carrigan, Emma; Dahyot, Rozenn
• Is Part Of: Computers & graphics. Volume 100(2021)
• Page Start: 81
• Page End: 92
• Publication Date: 2021-11
• Subjects: Computers and graphics -- Formatting -- Guidelines; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2021.07.022
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: The paper presents the first experiment on perception of perceptibility of Facial Action Units under different activation levels, which allows us to list the importance of blendshape expressions. Our main contribution is our perceptual models for perceptibility of facial action units and we provide our GitHub repository with the data and models in R-Code, allowing others to build on our data investigating a larger range of faces, viewpoints and facial action units, or to try different non-linear models such as neural networks. Graphical abstract: Abstract: Blendshape facial rigs are used extensively in the industry for facial animation of virtual humans. However, storing and manipulating large numbers of facial meshes (blendshapes) is costly in terms of memory and computation for gaming applications. Blendshape rigs are comprised of sets of semantically-meaningful expressions, which govern how expressive the character will be, often based on Action Units from the Facial Action Coding System (FACS). However, the relative perceptual importance of blendshapes has not yet been investigated. Research in Psychology and Neuroscience has shown that our brains process faces differently than other objects so we postulate that the perception of facial expressions will be feature-dependent rather than based purely on the amount of movement required to make the expression. Therefore, we believe that perception of blendshape visibility will not be reliably predicted by numerical calculations of the difference between the expression and the neutral mesh. In this paper, we explore the noticeability of blendshapes under different activation levels, and present new perceptually-based models to predict perceptual importance of blendshapes. The models predict visibility based on commonly-used geometry and image-based metrics.