Generative multiview inpainting for object removal in large indoor spaces. (9th March 2021)

• Record Type: Journal Article
• Authors: Kim, Joohyung; Hyeon, Janghun; Doh, Nakju
• Is Part Of: International journal of advanced robotic systems. Volume 18:Number 2(2021)
• Publication Date: 2021-03-09
• Subjects: Multiview inpainting -- object removal -- generative adversarial network -- color consistency -- boundary consistency; Robotics -- Periodicals; Robotics; Periodicals; 629.892
• DOI: 10.1177/1729881421996544
• Languages: English
• ISSNs: 1729-8806

Abstract

As interest in image-based rendering increases, the need for multiview inpainting is emerging. Despite of rapid progresses in single-image inpainting based on deep learning approaches, they have no constraint in obtaining color consistency over multiple inpainted images. We target object removal in large-scale indoor spaces and propose a novel pipeline of multiview inpainting to achieve color consistency and boundary consistency in multiple images. The first step of the pipeline is to create color prior information on masks by coloring point clouds from multiple images and projecting the colored point clouds onto the image planes. Next, a generative inpainting network accepts a masked image, a color prior image, imperfect guideline, and two different masks as inputs and yields the refined guideline and inpainted image as outputs. The color prior and guideline input ensure color and boundary consistencies across multiple images. We validate our pipeline on real indoor data sets quantitatively using consistency distance and similarity distance, metrics we defined for comparing results of multiview inpainting and qualitatively.