TriHorn-Net: A model for accurate depth-based 3D hand pose estimation. (1st August 2023)

• Record Type: Journal Article
• Authors: Rezaei, Mohammad; Rastgoo, Razieh; Athitsos, Vassilis
• Is Part Of: Expert systems with applications. Volume 223(2023)
• Publication Date: 2023-08-01
• Subjects: 3D hand pose estimation -- Depth image -- 2D joint -- Accuracy; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2023.119922
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: 3D hand pose estimation methods have made significant progress recently. However, estimation accuracy is often far from sufficient for specific real-world applications, and thus there is significant room for improvement. This paper proposes TriHorn-Net, a novel model that uses specific innovations to improve hand pose estimation accuracy on depth images. The first innovation is decomposition of the 3D hand pose estimation into the estimation of 2D joint locations in the depth image space (UV), and the estimation of their corresponding depths aided by two complementary attention maps. This decomposition prevents depth estimation, which is a more difficult task, from interfering with the UV estimations at both the prediction and feature levels. The second innovation is PixDropout, which is, to the best of our knowledge, the first appearance-based data augmentation method for hand depth images. Experimental results demonstrate that the proposed model outperforms the state-of-the-art methods on three public benchmark datasets, achieving 5.73 mm, 7.13 mm, and 7.68 mm on the ICVL, MSRA, and NYU datasets, respectively. The proposed model achieves this performance with a relatively fast average inference speed of 9.25 ms per frame on an NVIDIA 1080Ti GPU, which, as discussed in the paper, places the proposed model among the fastest-performing 3D hand pose estimation models. Our implementation is available at https://github.com/mrezaei92/TriHorn-Net . Highlights: 3D hand pose estimation has made significant progress recently. We propose a model, TriHorn-Net, for accurate depth-based 3D hand pose estimation. Our model, decomposes the 3D hand pose into the 2D joint locations. The proposed model uses the Pix-Dropout for appearance-based data augmentation. The proposed model outperforms the state-of-the-art methods on three public datasets.