A sparsity-promoting image decomposition model for depth recovery. (November 2020)

• Record Type: Journal Article
• Authors: Ye, Xinchen; Zhang, Mingliang; Yang, Jingyu; Fan, Xin; Guo, Fangfang
• Is Part Of: Pattern recognition. Volume 107(2020:Nov.)
• Publication Date: 2020-11
• Subjects: Image decomposition -- Depth recovery -- Depth discontinuities -- Depth cameras; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107506
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a general depth recovery model from signal decomposition perspective to better portray the 2D depth signal intrinsically. We proposed a new non-convex penalty to promote the prior sparsity and simultaneously maintain the convexity of the whole model for each variable. We introduce an iterative reweighted strategy to deal with the depth-color inconsistent problem and locate the depth boundaries. Experimental results demonstrate that the proposed method achieves promising performance in terms of recovery accuracy and running time Abstract: This paper proposes a novel image decomposition model for scene depth recovery from low-quality depth measurements and its corresponding high resolution color image. Through our observation, the depth map mainly contains smooth regions separated by additive step discontinuities, and can be simultaneously decomposed into a local smooth surface and an approximately piecewise constant component. Therefore, the proposed unified model combines the least square polynomial approximation (for smooth surface) and a sparsity-promoting prior (for piecewise constant) to better portray the 2D depth signal intrinsically. As we know, the representation of the piecewise constant signal in gradient domain is extremely sparse. Previous researches using total variation filter based on L 1 -norm or Lp -norm (0 < p < 1) are both sub-optimal when addressing the tradeoff between enhancing the sparsity and keeping the model convex. We propose a novel non-convex penalty based on Moreau envelope, which promotes the prior sparsity and simultaneously maintains the convexity of the whole model for each variable. We prove the convexity of the proposed model and give the convergence analysis of the algorithm. We also introduce an iterative reweighted strategy applied on the sparsity prior to deal with the depth-color inconsistent problem and to locate the depth boundaries. Moreover, we provide an accelerated algorithm to deal with the problem of non-uniform down-sampling when transforming the depth observation matrix into the Fourier domain for fast processing. Experimental results demonstrate that the proposed method can handle various types of depth degradation and achieve promising performance in terms of recovery accuracy and running time.