Exploiting Repetitions for Image‐Based Rendering of Facades. (20th July 2018)

• Record Type: Journal Article
• Authors: Rodriguez, Simon; Bousseau, Adrien; Durand, Fredo; Drettakis, George
• Is Part Of: Computer graphics forum. Volume 37:Number 4(2018)
• Page Start: 119
• Page End: 131
• Publication Date: 2018-07-20
• Subjects: CCS Concepts -- Computing methodologies → Computer graphics -- Image‐based rendering -- Reconstruction; Computer graphics -- Periodicals; 006.605
• DOI: 10.1111/cgf.13480
• Languages: English
• ISSNs: 0167-7055

Abstract

Abstract: Street‐level imagery is now abundant but does not have sufficient capture density to be usable for Image‐Based Rendering (IBR) of facades. We present a method that exploits repetitive elements in facades ‐ such as windows ‐ to perform data augmentation, in turn improving camera calibration, reconstructed geometry and overall rendering quality for IBR. The main intuition behind our approach is that a few views of several instances of an element provide similar information to many views of a single instance of that element. We first select similar instances of an element from 3–4 views of a facade and transform them into a common coordinate system, creating a "platonic" element. We use this common space to refine the camera calibration of each view of each instance and to reconstruct a 3D mesh of the element with multi‐view stereo, that we regularize to obtain a piecewise‐planar mesh aligned with dominant image contours. Observing the same element under multiple views also allows us to identify reflective areas ‐ such as glass panels ‐ which we use at rendering time to generate plausible reflections using an environment map. Our detailed 3D mesh, augmented set of views, and reflection mask enable image‐based rendering of much higher quality than results obtained using the input images directly.