Single image based 3D human pose estimation via uncertainty learning. (December 2022)

• Record Type: Journal Article
• Authors: Han, Chuchu; Yu, Xin; Gao, Changxin; Sang, Nong; Yang, Yi
• Is Part Of: Pattern recognition. Volume 132(2022)
• Publication Date: 2022-12
• Subjects: Uncertainty -- 3D pose estimation -- Graph convolutional network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108934
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose an uncertainty-based framework for 3D human pose estimation, which predicts 3D joint coordinates and uncertainties simultaneously. We develop an uncertainty-aware scaling factor to reshape the optimization objective. This manner improves the convergence speed and accuracy. We introduce a UAGCN to refine the initially estimated locations according to the estimated uncertainties. Experiments show the effectiveness of our approach quantitatively and qualitatively. In noisy scenes, our model exhibits greater robustness. Abstract: In monocular image scenes, 3D human pose estimation exhibits inherent ambiguity due to the loss of depth information and occlusions. Simply regressing body joints with high uncertainties will lead to model overfitting and poor generalization. In this paper, we propose an uncertainty-based framework to jointly learn 3D human poses and the uncertainty of each joint. Our proposed joint estimation framework aims to mitigate the adverse effects of training samples with high uncertainties and facilitate the training procedure. To be specific, we model each body joint as a Laplace distribution for uncertainty representation. Since visual joints often exhibit low uncertainties while occluded ones have high uncertainties, we develop an adaptive scaling factor, named the uncertainty-aware scaling factor, to ease the network optimization in accordance with the joint uncertainties. By doing so, our network is able to converge faster and significantly reduce the adverse effects caused by those ambiguous joints. Furthermore, we present an uncertainty-aware graph convolutional network by exploiting the learned joint uncertainties and the relationships among joints to refine the initial joint localization. Extensive experiments on single-person (Human3.6M) and multi-person (MuCo-3DHP & MuPoTS-3D) 3D human pose estimation datasets demonstrate the effectiveness of our method.