SMPLR: Deep learning based SMPL reverse for 3D human pose and shape recovery. (October 2020)

• Record Type: Journal Article
• Authors: Madadi, Meysam; Bertiche, Hugo; Escalera, Sergio
• Is Part Of: Pattern recognition. Volume 106(2020:Oct.)
• Publication Date: 2020-10
• Subjects: Deep learning -- 3D Human pose -- Body shape -- SMPL -- Denoising autoencoder -- Volumetric stack hourglass; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107472
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We use sparse body surface landmarks to represent the body shape and disambiguate joints orientations. We train a deep neural network to estimate the body pose and a detailed mesh without any special constraints on the skinned model parameters. Estimating the detailed mesh from 3D locations of joints and sparse landmarks is more accurate than estimating them directly from RGB image. We propose an efficient incremental and end-to-end training where a part of the model is a regularizer to the other part. Our simple modifications to a volumetric stacked hourglass network along with the proposed end-to-end training shows significant improvement over state-of-the-art human 3D pose estimation. Abstract: In this paper we propose to embed SMPL within a deep-based model to accurately estimate 3D pose and shape from a still RGB image. We use CNN-based 3D joint predictions as an intermediate representation to regress SMPL pose and shape parameters. Later, 3D joints are reconstructed again in the SMPL output. This module can be seen as an autoencoder where the encoder is a deep neural network and the decoder is SMPL model. We refer to this as SMPL reverse (SMPLR). By implementing SMPLR as an encoder-decoder we avoid the need of complex constraints on pose and shape. Furthermore, given that in-the-wild datasets usually lack accurate 3D annotations, it is desirable to lift 2D joints to 3D without pairing 3D annotations with RGB images. Therefore, we also propose a denoising autoencoder (DAE) module between CNN and SMPLR, able to lift 2D joints to 3D and partially recover from structured error. We evaluate our method on SURREAL and Human3.6M datasets, showing improvement over SMPL-based state-of-the-art alternatives by about 4 and 12 mm, respectively.