Semantic modeling of indoor scenes with support inference from a single photograph. (21st May 2018)

• Record Type: Journal Article
• Authors: Nie, Yinyu; Chang, Jian; Chaudhry, Ehtzaz; Guo, Shihui; Smart, Andi; Zhang, Jian Jun
• Is Part Of: Computer animation and virtual worlds. Volume 29:Number 3/4(2018)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2018-05-21
• Subjects: fully convolutional network -- indoor scene reconstruction -- semantic modeling -- support inference; Computer animation -- Periodicals; Visualization -- Periodicals; 006.6
• DOI: 10.1002/cav.1825
• Languages: English
• ISSNs: 1546-4261

Abstract

Abstract: We present an automatic approach for the semantic modeling of indoor scenes based on a single photograph, instead of relying on depth sensors. Without using handcrafted features, we guide indoor scene modeling with feature maps extracted by fully convolutional networks. Three parallel fully convolutional networks are adopted to generate object instance masks, a depth map, and an edge map of the room layout. Based on these high‐level features, support relationships between indoor objects can be efficiently inferred in a data‐driven manner. Constrained by the support context, a global‐to‐local model matching strategy is followed to retrieve the whole indoor scene. We demonstrate that the proposed method can efficiently retrieve indoor objects including situations where the objects are badly occluded. This approach enables efficient semantic‐based scene editing. Abstract : This work is aimed at retreiving 3D semantic contents of indoor rooms for scene understanding. With a single indoor photograph (a), three parallel fully convolutional networks are adopted to extract instance masks, a depth map and a layout edge map for support inference (b). Using the support relationships as constraints, a whole semantic indoor scene (c) can be automatically reconstructed.