A Hybrid Method of Cascaded Features for RGB-D Semantic Segmentation. Issue 1 (February 2021)

• Record Type: Journal Article
• Authors: Wang, Chenxu; Jiao, Jichao; Li, Ning; Deng, Zhongliang; Xu, Wei
• Is Part Of: Journal of physics. Volume 1792:Issue 1(2021)
• Publication Date: 2021-02
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/1792/1/012006
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: Fully Convolution Network and its following works has achieved the state-of-art performance on the task of RGB semantic segmentation. However, there still lacks an effective method to sufficiently leverage geometric information of depth image to accomplish RGB-D semantic segmentation. To this end, this paper proposed a new method containing two parts: 1) a simple but useful way based on Fast Marching Method to inpaint area of no-measured-depth pixels, which produces better result than standard dataset we used in this paper; 2) a new fusion architecture of CNN in which we fuse RGB and depth features in shallow layers to make geometry information from depth image a better assistant to RGB semantic segmentation. Besides, we add a feature filter architecture to help model choose the most discriminative features and punish useless or repeating features for better hierarchical expression. The original RGB image and inpainted depth image are fed into two feature extracting streams, one for each modality, and get fused in the fusion layer which is consecutively followed by deeper encoder network and decoder network. We evaluate our method on SUN RGB-D and NYUD dataset and experimental result shows that the proposed model has priority on RGB-D semantic segmentation.