Timid semi–supervised learning for face expression analysis. (June 2023)

• Record Type: Journal Article
• Authors: Badea, Mihai; Florea, Corneliu; Racoviţeanu, Andrei; Florea, Laura; Vertan, Constantin
• Is Part Of: Pattern recognition. Volume 138(2023)
• Publication Date: 2023-06
• Subjects: Face expression -- Action units -- Semi–supervised learning -- Diversity; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109417
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Face expression analysis suffers from insufficiently annotated data. We introduce the timid semi-supervised strategy based on diversity for face expression analysis. The solution produces superior results in both expression recognition and action unit estimation. Simple solutions based on diversity perform better when biased unlabelled data is available. Graphical abstract: Abstract: In the last years, semi–supervised learning has been proposed as a strategy with high potential for improving machine learning capabilities. Face expression recognition may highly benefit from such a technique, as accurate labeling is both difficult and costly, whereas millions of unlabeled images with human faces are available on the Internet, but without annotations. In this paper we evaluate the benefits of semi–supervised learning in the practical scenarios of face expression analysis. Our conclusion is that better performance is indeed achievable, but by methods that put a distinct emphasis on the diversity of exploring patterns in the unlabeled data domain. The evaluation is carried on multiple tasks such as detecting Action Units on EmotioNet, assessing Action Units intensity on the spontaneous DISFA database and, respectively, recognizing expressions on static images acquired in the wild, from the RAF-DB and FER+ databases. We show that, in these scenarios, a so–called timid semi–supervised learner is more robust and achieves higher performance than standard, confident semi–supervised learners.