Task-dependent multi-task multiple kernel learning for facial action unit detection. (March 2016)

• Record Type: Journal Article
• Authors: Zhang, Xiao; Mahoor, Mohammad H.
• Is Part Of: Pattern recognition. Volume 51(2016:Mar.)
• Page Start: 187
• Page End: 196
• Publication Date: 2016-03
• Subjects: Facial action unit detection -- Multi-task multiple kernel learning -- Support vector machines; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2015.08.026
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: Facial action unit (AU) detection from images and videos is a challenging research topic and has attracted great attention in the past few years. This paper presents a novel method, task-dependent multi-task multiple kernel learning (TD-MTMKL), to jointly detect the absence and presence of multiple AUs. TD-MTMKL attempts to learn an optimal kernel combination from a given set of basis kernels for each involved AU and obtain a finer depiction of AU relations through kernel combination weights. In other words, AU detection is solved as a multi-task multiple kernel learning problem, where AU relations are encoded via their SVM discriminative hyperplanes and kernel combination weights. The kernel learning increases the discriminant power of the classifier by fusing different types of facial feature representations with multiple kernels. Specifically, based on the TD-MTMKL method proposed in this paper, co-occurrence AUs share the same kernel weights while AUs with weak co-occurrence relations may employ distinct sets of kernels. Such "task-dependent" kernel learning framework seeks a trade-off between capturing commonalities and adapting to variations in modeling AU relations. Our experiments on the CK+ and DISFA databases show that our method achieved encouraging detection results of both post and spontaneous AUs compared to the state-of-the-art methods. Abstract : Highlights: AU co-occurrence relations are modeled as multi-task multiple kernel learning. TD-MTMKL fuses different facial feature representations with multiple kernels. TD-MTMKL captures commonalities and adapts to variations among co-occurred AUs.