Reliability and validity of machine vision for the assessment of facial expressions. (August 2019)

• Record Type: Journal Article
• Authors: Beringer, Matthias; Spohn, Frank; Hildebrandt, Andrea; Wacker, Jan; Recio, Guillermo
• Is Part Of: Cognitive systems research. Volume 56(2019)
• Page Start: 119
• Page End: 132
• Publication Date: 2019-08
• Subjects: Reliability -- Validity -- FACET -- Facial expression -- Facial action coding system; Cognition -- Periodicals; Cognitive engineering (System design) -- Periodicals; Artificial intelligence -- Periodicals; 153.05
• DOI: 10.1016/j.cogsys.2019.03.009
• Languages: English
• ISSNs: 1389-0417

Abstract

Graphical abstract: Highlights: We observe reliable assessment of facial expressions with machine vision. Assessment of expressions of emotions are slightly more accurate than of action units. Low resolution of the video input impacts reliability the most. Onset of smiles and frowns detected with machine vision converge with electromyogram. Abstract: Automated assessment of facial expressions with machine vision software opens up new opportunities for the assessment of facial expression in a shrewd and economic way in psychological and applied research. We investigated the assessment quality of one machine vision algorithm (FACET) in a study using standardized databases of dynamic facial expressions in different conditions (angle, distance, lighting and resolution). We found high reliability in terms of ratings concordance across conditions for facial expressions (intraclass correlation, ICC = 0.96) and action units (ICC = 0.78). Signal detection analyses showed good classification for both facial expressions (area under the curve, AUC > 0.99) and action unit scores (AUC = 0.91). In a second study, we investigated the convergent validity of machine vision assessment and electromyography (EMG) with regard to reaction times measured during the production of smiles (action unit 12) and frowns (action unit 4). To this end, we simultaneously measured EMG and expression classification with machine vision software in a response priming task with validly and invalidly primed responses. Both, EMG and machine vision data revealed similar performance costs in reaction times of inhibiting the falsely prepared expression and reprogramming the correct one. These results support machine vision as a suitable tool for assessing experimental effects in facial reaction times.