Variational method for joint optical flow estimation and edge-aware image restoration. (May 2017)

• Record Type: Journal Article
• Authors: Tu, Zhigang; Xie, Wei; Cao, Jun; van Gemeren, Coert; Poppe, Ronald; Veltkamp, Remco C.
• Is Part Of: Pattern recognition. Volume 65(2017:May)
• Page Start: 11
• Page End: 25
• Publication Date: 2017-05
• Subjects: Optical flow -- Image sequence restoration -- Edge preserving -- Efficient numerical solver; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2016.10.027
• Languages: English
• ISSNs: 0031-3203

Abstract

Abstract: The most popular optical flow algorithms rely on optimizing the energy function that integrates a data term and a smoothness term. In contrast to this traditional framework, we derive a new objective function that couples optical flow estimation and image restoration. Our method is inspired by the recent successes of edge-aware constraints (EAC) in preserving edges in general gradient domain image filtering. By incorporating an EAC image fidelity term (IFT) in the conventional variational model, the new energy function can simultaneously estimate optical flow and restore images with preserved edges, in a bidirectional manner. For the energy minimization, we rewrite the EAC into gradient form and optimize the IFT with Euler–Lagrange equations. We can thus apply the image restoration by analytically solving a system of linear equations. Our EAC-combined IFT is easy to implement and can be seamlessly integrated into various optical flow functions suggested in literature. Extensive experiments on public optical flow benchmarks demonstrate that our method outperforms the current state-of-the-art in optical flow estimation and image restoration. Abstract : Highlights: Incorporating an EAC added IFT to the variational model to form a new energy function, which can estimate optical flow and restore images jointly. The EAC can be rewritten into the first-order gradient form, and is beneficial for preserving edges and minimization. Input images can be fast restored by optimizing the Euler–Lagrange equations of the EAC integrated IFT.