Detecting deception through facial expressions in a dataset of videotaped interviews: A comparison between human judges and machine learning models. (February 2022)

• Record Type: Journal Article
• Authors: Monaro, Merylin; Maldera, Stéphanie; Scarpazza, Cristina; Sartori, Giuseppe; Navarin, Nicolò
• Is Part Of: Computers in human behavior. Volume 127(2022)
• Publication Date: 2022-02
• Subjects: Facial micro-expressions -- Deception detection -- Machine learning -- Deep learning -- Low-stakes lies; Interactive computer systems -- Periodicals; Man-machine systems -- Periodicals; 004.019
• DOI: 10.1016/j.chb.2021.107063
• Languages: English
• ISSNs: 0747-5632

Abstract

Abstract: In the last decades, research has claimed facial micro-expressions are a reliable means to detect deception. However, experimental results showed that trained and naïve human judges observing facial micro-expressions can distinguish liars from truth-tellers with an accuracy just slightly above the chance level. More promising results recently came from the field of artificial intelligence, in which machine learning (ML) techniques are used to identify micro-expressions and are trained to distinguish deceptive statements from genuine ones. In this paper, we test the ability of different feature extraction methods (i.e., improved dense trajectories, OpenFace) and ML techniques (i.e., support vector machines vs. deep neural networks) to distinguish liars from truth-tellers based on facial micro-expressions, using a new video data set collected in low-stakes situations. During the interviews, a technique to increase liars' cognitive load was applied, facilitating cues of lies to emerge. Results highlighted that support vector machines (SVMs) coupled with OpenFace resulted in the best performing method (AUC = 0.72 videos without cognitive load; AUC = 0.78 videos with cognitive load). All the tested classifiers performed better when a cognitive load was imposed on the interviewee, confirming that the technique of increasing cognitive load during an interview facilitates deception recognition. In the same task, human judges obtained an accuracy of 57%. Results are discussed and compared with the previous literature, confirming that artificial intelligence performs better than humans in lie-detection tasks do, even when humans have more information to make a decision. Highlights: Automatic systems are better than humans in detecting deception from videos. AI is a promising methodology to detect deception through facial micro-expressions. AI performs better when the cognitive load of the interviewees is increased. SVM is more able in distinguish liars from truth-tellers when trained with AUs as features.