3D pose estimation and future motion prediction from 2D images. (April 2022)

• Record Type: Journal Article
• Authors: Yang, Ji; Ma, Youdong; Zuo, Xinxin; Wang, Sen; Gong, Minglun; Cheng, Li
• Is Part Of: Pattern recognition. Volume 124(2022)
• Publication Date: 2022-04
• Subjects: Pose estimation -- Motion prediction -- Multitask learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108439
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Estimating 3D human body poses and predicting future 3D motions are jointly investigated with a novel image-grounded multitask learning framework. Lie algebra pose representation implicitly encodes strong physical constraints for estimating poses and motions of articulated objects. Accurate and realistic motion prediction can be obtained from 2D images and no long rely on 3D motion capture data input. Abstract: This paper considers to jointly tackle the highly correlated tasks of estimating 3D human body poses and predicting future 3D motions from RGB image sequences. Based on Lie algebra pose representation, a novel self-projection mechanism is proposed that naturally preserves human motion kinematics. This is further facilitated by a sequence-to-sequence multi-task architecture based on an encoder-decoder topology, which enables us to tap into the common ground shared by both tasks. Finally, a global refinement module is proposed to boost the performance of our framework. The effectiveness of our approach, called PoseMoNet, is demonstrated by ablation tests and empirical evaluations on Human3.6M and HumanEva-I benchmark, where competitive performance is obtained comparing to the state-of-the-arts.