Spatial feature mapping for 6DoF object pose estimation. (November 2022)

• Record Type: Journal Article
• Authors: Mei, Jianhan; Jiang, Xudong; Ding, Henghui
• Is Part Of: Pattern recognition. Volume 131(2022)
• Publication Date: 2022-11
• Subjects: 6D Pose estimation -- Rotation symmetry -- Spherical convolution -- Graph convolutional network; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108835
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We explore the possibility of solving the rotation ambiguous and occlusion problems in 6D pose estimation by using the spherical correlation and graph convolution. A robust 6D object pose estimation system is proposed. We propose to map the 2D convolutional feature to both a sphere and the 3D mesh representation. And the corresponding network components are integrated which forms the end-to-end deep neural network training and inference scheme. The proposed method digs more essential information from the data representation and processing. To our knowledge, this is new and meaningful for 6D object pose estimation. Target on the rotation ambiguous and occlusion challenges, we propose to solve from the parameter space and data structure. The state-of-the-art performance demonstrates the efficiency of the proposed system. Abstract: This work aims to estimate 6Dof (6D) object pose in background clutter. Considering the strong occlusion and background noise, we propose to utilize the spatial structure for better tackling this challenging task. Observing that the 3D mesh can be naturally abstracted by a graph, we build the graph using 3D points as vertices and mesh connections as edges. We construct the corresponding mapping from 2D image features to 3D points for filling the graph and fusion of the 2D and 3D features. Afterward, a Graph Convolutional Network (GCN) is applied to help the feature exchange among objects' points in 3D space. To address the problem of rotation symmetry ambiguity for objects, a spherical convolution is utilized and the spherical features are combined with the convolutional features that are mapped to the graph. Predefined 3D keypoints are voted and the 6DoF pose is obtained via the fitting optimization. Two scenarios of inference, one with the depth information and the other without it are discussed. Tested on the datasets of YCB-Video and LINEMOD, the experiments demonstrate the effectiveness of our proposed method.