A CNN-based 3D human pose estimation based on projection of depth and ridge data. (October 2020)

• Record Type: Journal Article
• Authors: Kim, Yeonho; Kim, Daijin
• Is Part Of: Pattern recognition. Volume 106(2020:Oct.)
• Publication Date: 2020-10
• Subjects: 3D Human pose estimation -- 3D Point projection -- Ridge data; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107462
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a CNN-based human pose estimation using depth and ridge data. We project the depth and ridge data on three orthogonal planes (XY, XZ, ZY). The projected depth and ridge data can eliminate the 3D information loss. Ridge data is introduced to avoid joint drift, which improves the accuracy of estimated poses. The proposed method achieved the state-of-the-art pose estimation accuracies. Abstract: We propose a method that use a convolutional neural network (CNN) to estimate human pose by analyzing the projection of the depth and ridge data, which represent local maxima in a distance transform map. To fully utilize the 3D information of depth points, we propose a method to project the depth and ridge data on various directions. The proposed projection method can reduce the 3D information loss, the ridge data can avoid joint drift, and the CNN increases localization accuracy. The proposed method proceeds as follows. (1) We use depth data to segment the human from the background and extract ridge data from human silhouettes. (2) We project the depth and ridge data onto XY, XZ, and ZY planes. (3) ResNet-101 accepts six projected images and use 1 × 1 convolution layers to generate 2D heatmaps and offsets. (4) We generate 2D keypoints per plane by using the soft-argmax operation. (5) We obtain 3D joint positions by using the fully-connected layers. In experiments on the SMMC-10, EVAL, and ITOP datasets, the proposed method achieved the state-of-the-art pose estimation accuracies. The proposed method can eliminate the 3D information loss and drift of joint positions that can occur during estimation of human pose.