Single-shot 3D multi-person pose estimation in complex images. (April 2021)

• Record Type: Journal Article
• Authors: Benzine, Abdallah; Luvison, Bertrand; Pham, Quoc Cuong; Achard, Catherine
• Is Part Of: Pattern recognition. Volume 112(2021)
• Publication Date: 2021-04
• Subjects: Multi-person -- 3D -- Human pose -- Deep learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107534
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Multi-person 3D human pose estimation in rich and complex environments. 2D and 3D human joints are predicted using heatmaps and Occlusion Robust Pose Maps. The difficult problem of associating joints to people skeletons is managed using the associative embeddings method. The proposed approach results surpass single-shot methods of the state of the art on the CMU Panoptic dataset and MuPoTS-3D datasets. Experiments on the JTA Dataset show that complex urban scenarios with many people at different image resolution remains a challenge for our approach. Abstract: In this paper, we propose a new single shot method for multi-person 3D human pose estimation in complex images. The model jointly learns to locate the human joints in the image, to estimate their 3D coordinates and to group these predictions into full human skeletons. The proposed method deals with a variable number of people and does not need bounding boxes to estimate the 3D poses. It leverages and extends the Stacked Hourglass Network and its multi-scale feature learning to manage multi-person situations. Thus, we exploit a robust 3D human pose formulation to fully describe several 3D human poses even in case of strong occlusions or crops. Then, joint grouping and human pose estimation for an arbitrary number of people are performed using the associative embedding method. Our approach significantly outperforms the state of the art on the challenging CMU Panoptic and a previous single shot method on the MuPoTS-3D dataset. Furthermore, it leads to good results on the complex and synthetic images from the newly proposed JTA Dataset.