Learn to model blurry motion via directional similarity and filtering. (March 2018)

• Record Type: Journal Article
• Authors: Li, Wenbin; Chen, Da; Lv, Zhihan; Yan, Yan; Cosker, Darren
• Is Part Of: Pattern recognition. Volume 75(2018:Mar.)
• Page Start: 327
• Page End: 338
• Publication Date: 2018-03
• Subjects: Optical flow -- Convolutional Neural Network (CNN) -- Video/image deblurring -- Directional filtering; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.04.020
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A hybrid framework to extract optical flow from blurry footages. A CNN component for image deblurring. A learnable directional filtering layer encodes the angle and distance similarity between blur and image properties. Two synthetic ground truth sequences for the blurry scenes. Abstract: It is difficult to recover the motion field from a real-world footage given a mixture of camera shake and other photometric effects. In this paper we propose a hybrid framework by interleaving a Convolutional Neural Network (CNN) and a traditional optical flow energy. We first conduct a CNN architecture using a novel learnable directional filtering layer. Such layer encodes the angle and distance similarity matrix between blur and camera motion, which is able to enhance the blur features of the camera-shake footages. The proposed CNNs are then integrated into an iterative optical flow framework, which enable the capability of modeling and solving both the blind deconvolution and the optical flow estimation problems simultaneously. Our framework is trained end-to-end on a synthetic dataset and yields competitive precision and performance against the state-of-the-art approaches.