A dense flow-based framework for real-time object registration under compound motion. (March 2017)

• Record Type: Journal Article
• Authors: Yang, Songfan; An, Le; Lei, Yinjie; Li, Mingyang; Thakoor, Ninad; Bhanu, Bir; Liu, Yiguang
• Is Part Of: Pattern recognition. Volume 63(2017:Mar.)
• Page Start: 279
• Page End: 290
• Publication Date: 2017-03
• Subjects: Object registration -- Spontaneous facial expression -- SIFT flow -- Optical flow -- Super-resolution; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.10.015
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: A moving object often has elastic and deformable surfaces (e.g., a human head). Tracking and measuring surface deformation while the object itself is also moving is a challenging, yet important problem in many video analysis tasks. For example, video-based facial expression recognition requires tracking non-rigid motions of facial features without being affected by any rigid motions of the head. In this paper, we present a generic video alignment framework to extract and characterize surface deformations accompanied by rigid-body motions with respect to a fixed reference (a canonical form). We propose a generic model for object alignment in a Bayesian framework, and rigorously show that a special case of the model results in a SIFT flow and optical flow based least-square problem. We demonstrate that dynamic programming can be used to speed up the computation of our algorithm. The proposed algorithm is evaluated on three applications, including the analysis of subtle facial muscle dynamics in spontaneous expressions, face image super-resolution, and generic object registration. Experimental results, in terms of both qualitative and quantitative measures, demonstrate the efficacy of the proposed algorithm, which can be executed in real time. Abstract : Highlights: A moving object often has elastic and deformable surfaces (e.g., a human head). Measuring surface deformation while the object itself is also moving is challenging. We present a generic video alignment framework to characterize surface deformations. The proposed algorithm is suitable for spontaneous expressions recognition. It also improves the performance of face image super-resolution.