Micro‐expression recognition by two‐stream difference network. Issue 6 (30th March 2021)

• Record Type: Journal Article
• Authors: Pan, Hang; Xie, Lun; Li, Juan; Lv, Zeping; Wang, Zhiliang
• Is Part Of: IET computer vision. Volume 15:Issue 6(2021)
• Page Start: 440
• Page End: 448
• Publication Date: 2021-03-30
• Subjects: Computer vision -- Periodicals; Pattern recognition systems -- Periodicals; 006.37
• DOI: 10.1049/cvi2.12030
• Languages: English
• ISSNs: 1751-9632

Abstract

Abstract: Facial micro‐expression is a superposition of micro‐expression features and identity information of a subject. For the problem of identity information interference in micro‐expression recognition, this study proposes a new method for facial micro‐expression recognition by de‐identity information, called two‐stream difference network (TSDN). First, a two‐stream encoder‐decoder network is trained by a convolutional neural network, where the input of the micro‐expression stream is a micro‐expression image, and the identity stream is a facial identity image. The micro‐expression image is the apex image, and the identity image is the onset image in the micro‐expression sequence. The identity information and micro‐expression features are recorded in the intermediate layer of the micro‐expression stream, while the intermediate layer of the identity stream contains only the identity information of a subject. Then, the identity information is removed by the difference network, but micro‐expression features are stored in the intermediate layer of the micro‐expression stream. Given the sequence of the micro‐expressions, the TSDN model of de‐identity information learns the difference that stores in the expression stream. Two public spontaneous facial micro‐expression data sets (SMIC and CASME II) are employed in our experiments. The experiment results show that our model can achieve a superior performance in micro‐expression recognition.