Multi-Task Joint Learning of 3D Keypoint Saliency and Correspondence Estimation. (December 2021)

• Record Type: Journal Article
• Authors: Wei, Guangshun; Ma, Long; Wang, Chen; Desrosiers, Christian; Zhou, Yuanfeng
• Is Part Of: Computer aided design. Volume 141(2021)
• Publication Date: 2021-12
• Subjects: Keypoint saliency estimation -- Keypoint correspondence estimation -- Votes -- Confidence map -- Model matching; Computer-aided design -- Periodicals; Engineering design -- Data processing -- Periodicals; Computer graphics -- Periodicals; Conception technique -- Informatique -- Périodiques; Infographie -- Périodiques; Computer graphics; Engineering design -- Data processing; Periodicals; Electronic journals; 620.00420285
• DOI: 10.1016/j.cad.2021.103105
• Languages: English
• ISSNs: 0010-4485

Abstract

Abstract: 3D keypoint detection is an essential problem in computer graphics and computer vision, especially for 3D shape analysis and model matching. In this paper, we propose a novel multi-task joint learning network architecture for 3D keypoint saliency estimation and correspondence estimation. To better capture the local and global features of the 3D model, we design a spatial multi-scale perception module that concatenates feature maps at different scales during the extraction of point cloud features. In the multi-task joint learning process, we obtain the offset vector of each point to the keypoint in the 3D model through a voting mechanism. This mechanism predicts the confidence value of each point in the 3D model and then filters out low-confidence points to generate a reliable voting result. Afterwards, keypoint saliency estimation is achieved through clustering. In parallel, keypoint correspondence estimation is learned by predicting the semantic label of the selected high-confidence points. In extensive evaluations, ablation studies and comparisons, we demonstrate that the proposed architecture can both efficiently and accurately detect the position and semantic labels of the 3D keypoints, which enables it to outperform state-of-the-art approaches for 3D keypoint detection. Graphical abstract: Highlights: A novel multi-task joint learning network is presented for estimating keypoint saliency and correspondence simultaneously on 3D point cloud. The spatial multi-scale perception module effectively combines both local details and global semantic information to achieve higher performance in the multi-task prediction. A confidence-aware strategy improves the performance of keypoint saliency and correspondence estimation by clustering and classifying points with higher prediction confidence.