Automatic layered RGB‐D scene flow estimation with optical flow field constraint. Issue 16 (23rd February 2021)

• Record Type: Journal Article
• Authors: Li, Xiuxiu; Liu, Yanjuan; Jin, Haiyan; Zheng, Jiangbin; Cai, Lei
• Is Part Of: IET image processing. Volume 14:Issue 16(2020)
• Page Start: 4092
• Page End: 4101
• Publication Date: 2021-02-23
• Subjects: computer vision -- image colour analysis -- image segmentation -- image sequences -- video signal processing -- edge detection; optical flow field constraint -- red blue green depth frames -- object segmentation -- incomplete segmented areas -- super‐pixel segmentation -- automatic layering -- automatic layered RGB‐D scene flow estimation -- RGB‐D frames -- digital video processing -- computer vision -- object occlusion -- depth image -- motion information -- edge preservation; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2020.0230
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : Scene flow estimation with RGB‐D frames is receiving increasing attention in digital video processing and computer vision due to the widespread use of depth sensors. Existing methods based on object segmentation have shown their effectiveness for object occlusion and large displacement. However, improper segmentation often causes incomplete segmented areas or incorrect edges, which will result in inaccurate occlusion inference and scene flow estimation. To this end, an automatic layered RGB‐D scene flow estimation method is proposed, which achieves more accurate layering of objects in depth image by exploring motion information. The authors exploit super‐pixel segmentation for initial layering, which is beneficial for preserving edges and integrity of the objects. Furthermore, an optical flow which is highly correlated with the scene is also used for automatic layering. Finally, the utilisation of super‐pixel segmentation and motion information would ensure the integrity of the object area and improve the accuracy of scene flow. They have validated their approach both qualitatively and quantitatively on several public datasets. Experimental results show that the proposed method is able to preserve the integrity of the object and achieve lower root mean square error and average angular error as compared with the current state‐of‐the‐art algorithms.