3D Layout encoding network for spatial‐aware 3D saliency modelling. Issue 5 (10th July 2019)

• Record Type: Journal Article
• Authors: Yuan, Jing; Cao, Yang; Kang, Yu; Song, Weiguo; Yin, Zhongcheng; Ba, Rui; Ma, Qing
• Is Part Of: IET computer vision. Volume 13:Issue 5(2019)
• Page Start: 480
• Page End: 488
• Publication Date: 2019-07-10
• Subjects: image sensors -- image fusion -- object detection -- image colour analysis -- feature extraction; popular 3D multimedia applications -- existing 3D devices -- low quality -- holes -- predictions -- single depth images -- deep layout features -- spatial-aware saliency prediction -- coarse depth-induced saliency cues -- depth details -- high-quality RGB image -- low-level -- final prediction -- spatial layout -- saliency modelling results; Computer vision -- Periodicals; Pattern recognition systems -- Periodicals; 006.37
• DOI: 10.1049/iet-cvi.2018.5591
• Languages: English
• ISSNs: 1751-9632

Abstract

Abstract : Three‐dimensional (3D) [red, green and blue (RGB) + depth] saliency modelling can help with popular 3D multimedia applications. However, depth images produced from existing 3D devices are often with low quality, e.g. containing noises and holes. In this study, rather than relying on features or predictions directly derived from single depth images, the authors propose to encode deep layout features to facilitate the spatial‐aware saliency prediction. Specifically, they first generate coarse depth‐induced saliency cues which are careless of depth details. Then, to leverage the information of the high‐quality RGB image, they embed both low‐level and high‐level RGB deep features to refine the final prediction. In this way, they take both bottom‐up and top‐down cues together with spatial layout into account and achieve better saliency modelling results. Experiments on five public datasets show the superiority of the proposed method.