Photorealistic Material Editing Through Direct Image Manipulation. (20th July 2020)

• Record Type: Journal Article
• Authors: Zsolnai‐Fehér, Károly; Wonka, Peter; Wimmer, Michael
• Is Part Of: Computer graphics forum. Volume 39:Number 4(2020)
• Page Start: 107
• Page End: 120
• Publication Date: 2020-07-20
• Subjects: CCS Concepts -- Computing methodologies → Neural networks -- Rendering -- Ray tracing; Computer graphics -- Periodicals; 006.605
• DOI: 10.1111/cgf.14057
• Languages: English
• ISSNs: 0167-7055

Abstract

Abstract: Creating photorealistic materials for light transport algorithms requires carefully fine‐tuning a set of material properties to achieve a desired artistic effect. This is typically a lengthy process that involves a trained artist with specialized knowledge. In this work, we present a technique that aims to empower novice and intermediate‐level users to synthesize high‐quality photorealistic materials by only requiring basic image processing knowledge. In the proposed workflow, the user starts with an input image and applies a few intuitive transforms (e.g., colorization, image inpainting) within a 2D image editor of their choice, and in the next step, our technique produces a photorealistic result that approximates this target image. Our method combines the advantages of a neural network‐augmented optimizer and an encoder neural network to produce high‐quality output results within 30 seconds. We also demonstrate that it is resilient against poorly‐edited target images and propose a simple extension to predict image sequences with a strict time budget of 1–2 seconds per image.