Discriminative unimodal feature selection and fusion for RGB-D salient object detection. (February 2022)

• Record Type: Journal Article
• Authors: Huang, Nianchang; Luo, Yongjiang; Zhang, Qiang; Han, Jungong
• Is Part Of: Pattern recognition. Volume 122(2022)
• Publication Date: 2022-02
• Subjects: RGB-D salient object detection -- Discriminative unimodal feature selection -- Semantic information -- Multi-scale cross-modal feature fusion; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.108359
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel RGB-D salient object detection model, which takes the equalities of input into consideration. An SG-MWMG sub-network is designed to determine the informative and non-informative regions in the input RGB-D images. An MCFF module is designed to fuse the unimodal RGB and depth features at multiple scales and levels. Abstract: Most existing RGB-D salient object detectors make use of the complementary information of RGB-D images to overcome the challenging scenarios, e.g., low contrast, clutter backgrounds. However, these models generally neglect the fact that one of the input images may be poor in quality. This will adversely affect the discriminative ability of cross-modal features when the two channels are fused directly. To address this issue, a novel end-to-end RGB-D salient object detection model is proposed in this paper. At the core of our model is a Semantic-Guided Modality-Weight Map Generation (SG-MWMG) sub-network, producing modality-weight maps to indicate which regions on both modalities are high-quality regions, given input RGB-D images and the guidance of their semantic information. Based on it, a Bi-directional Multi-scale Cross-modal Feature Fusion (Bi-MCFF) module is presented, where the interactions of the features across different modalities and scales are exploited by using a novel bi-directional structure for better capturing cross-scale and cross-modal complementary information. The experimental results on several benchmark datasets verify the effectiveness and superiority of the proposed method over some state-of-the-art methods.