PASIFTNet: Scale-and-Directional-Aware Semantic Segmentation of Point Clouds. (March 2023)

• Record Type: Journal Article
• Authors: Wang, Shaofan; Liu, Ying; Wang, Lichun; Sun, Yanfeng; Yin, Baocai
• Is Part Of: Computer aided design. Volume 156(2023)
• Publication Date: 2023-03
• Subjects: Semantic segmentation -- Point-atrous convolution -- Scale-and-directional-aware; Computer-aided design -- Periodicals; Engineering design -- Data processing -- Periodicals; Computer graphics -- Periodicals; Conception technique -- Informatique -- Périodiques; Infographie -- Périodiques; Computer graphics; Engineering design -- Data processing; Periodicals; Electronic journals; 620.00420285
• DOI: 10.1016/j.cad.2022.103462
• Languages: English
• ISSNs: 0010-4485

Abstract

Abstract: Point clouds obey the sparsity, disorderliness and irregularity properties, leading to noisy or unrobust features during the 3D semantic segmentation task. Existing approaches cannot fully mine local geometry and context information of point clouds, due to their irrational feature learning or neighborhood selection schemes. In this paper, we propose a Point-Atrous SIFT Network (PASIFTNet) for learning multi-scale multi-directional features of point clouds. PASIFTNet is a hierarchical encoder–decoder network, which combines the Point-Atrous SIFT (PASIFT) modules and edge-preserved pooling/unpooling modules alternatively during the encoder/decoder stage. The key component of PASIFTNet is the Point-Atrous Orientation Encoding unit of the PASIFT module, which can arbitrarily expand its receptive fields to incorporate larger context information and extract scale-and-directional-aware feature point information, benefiting from the quadrant-wise SIFT-like point-atrous convolution. Moreover, the edge-preserved pooling/unpooling modules complement PASIFTNet by preserving the edge features and recovering the high-dimensional features of point clouds. We conduct experiments on two public 3D point cloud datasets: ScanNet, S3DIS and a real-world unlabeled dataset FARO-3 collected by the FARO laser scanner. The quantitative results show that, PASIFTNet achieves 86.8% overall accuracy on ScanNet and achieves 86.5% overall accuracy and 68.3% mean intersection-over-union on S3DIS . Moreover, PASIFTNet exhibits a satisfactory robustness and generalization ability towards unknown scenes on FARO-3 . Highlights: The Point-Atrous Orientation Encoding (PAOE) unit captures local neighborhood information comprehensively and enhances the point-wise features. We stack several PAOE units together to form the PASIFT module, which has scale awareness and portability. We propose an encoder-decoder network: PASIFTNet, which learns multi-scale multi-directional features of point clouds using an alternative architecture of PASIFT modules and EP/EU modules. Experiments on three point cloud datasets prove its effectiveness and universality against several state-of-the-art methods on the semantic segmentation task.