TreeNet: Structure preserving multi-class 3D point cloud completion. (July 2023)

• Record Type: Journal Article
• Authors: Xi, Long; Tang, Wen; Wan, TaoRuan
• Is Part Of: Pattern recognition. Volume 139(2023)
• Publication Date: 2023-07
• Subjects: 3D Point cloud completion -- Multi-class training -- Hierarchical tree -- Computer vision -- Artificial intelligence -- Deep learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2023.109476
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A novel TreeNet-multiclass decoder is proposed for multi-class 3D point cloud completion. A novel TreeNet-binary decoder is proposed, which focuses on generating points in missing areas and fully preserving the original partial input 3D point cloud. A novel TreeNet is proposed, which combines the advantages of the TreeNet-multiclass and the TreeNet-binary for 3D point cloud completion. Three novel forward and backward propagation methods are proposed to train TreeNet-multiclass, TreeNet-binary and TreeNet decoders, respectively. TreeNet-multiclass, TreeNet-binary and TreeNet exhibit strong generalization to unknown classes that are never trained. Graphical abstract: Abstract: Generating the missing data of 3D object point clouds from partial observations is a challenging task. Existing state-of-the-art learning-based 3D point cloud completion methods tend to use a limited number of categories/classes of training data and regenerate the entire point cloud based on the training datasets. As a result, output 3D point clouds generated by such methods may lose details (i.e. sharp edges and topology changes) due to the lack of multi-class training. These methods also lose the structural and spatial details of partial inputs due to the models do not separate the reconstructed partial input from missing points in the output. In this paper, we propose a novel deep learning network - TreeNet for 3D point cloud completion. TreeNet has two networks in hierarchical tree-based structures: TreeNet-multiclass focuses on multi-class training with a specific class of the completion task on each sub-tree to improve the quality of point cloud output; TreeNet-binary focuses on generating points in missing areas and fully preserving the original partial input. TreeNet-multiclass and TreeNet-binary are both network decoders and can be trained independently. TreeNet decoder is the combination of TreeNet-multiclass and TreeNet-binary and is trained with an encoder from existing methods (i.e. PointNet encoder). We compare the proposed TreeNet with five state-of-the-art learning-based methods on fifty classes of the public Shapenet dataset and unknown classes, which shows that TreeNet provides a significant improvement in the overall quality and exhibits strong generalization to unknown classes that are not trained.