RGB-D salient object ranking based on depth stack and truth stack for complex indoor scenes. (May 2023)

• Record Type: Journal Article
• Authors: Deng, Jingzheng; Zhang, Jinxia; Hu, Zewen; Wang, Liantao; Jiang, Jiacheng; Zhu, Xinchao; Chen, Xinyi; Yuan, Yin; Wang, Chao
• Is Part Of: Pattern recognition. Volume 137(2023)
• Publication Date: 2023-05
• Subjects: Complex scenes -- RGB-D -- Salient object ranking -- Indoor -- Depth; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.109251
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Firstly, we propose a new research problem to rank salient objects in the RGB-D saliency detection field. This research problem is inspired by the visual perception of humans, who shift their attention from one object to another. Secondly, we construct an RGB-D salient object ranking dataset that contains complex indoor images with multiple objects. We analyze the dataset in-depth and compare it with salient object detection datasets. Thirdly, we propose an end-to-end learning network that fully uses depth information based on depth stack and ground truth stack to perform RGBD salient object ranking tasks. Experimental comparisons demonstrate the effectiveness of the proposed method. Abstract: RGB-D salient object detection has achieved a great development in recent years due to its extensive applications. Previous studies mainly focus on simple scene images with one single object. These models usually become overwhelmed by complex scenes with multiple objects. Moreover, these methods model salient object detection as a binary segmentation problem. However, psychology studies show that humans shift their visual attention from one object to another and rank salient objects, especially in complex indoor scenes. Following the psychological studies, we propose to rank salient objects in RGB-D images of complex indoor scenes. Due to the lack of such data, we first construct a RGB-D salient object ranking dataset containing complex indoor scenes with multiple objects. The saliency ranking of different objects is defined based on the order that an observer notices these objects. The final salient object ranking result is an average across the saliency rankings of 13 observers. This RGB-D salient object ranking dataset is also analyzed with current mainstream RGB-D salient object detection dataset for comparison. Since location information provided by depth images can help to determine the saliency ranking of objects, we further propose an end-to-end network exploiting depth stack and ground truth stack to predict the order of salient objects in complex scenes. The quantitative and qualitative comparisons demonstrate the effectiveness of the proposed method.