Retrieving indoor objects: 2D-3D alignment using single image and interactive ROI-based refinement. (February 2018)

• Record Type: Journal Article
• Authors: Liu, Fuchang; Wang, Shuangjian; Ding, Dandan; Yuan, Qingshu; Yao, Zhengwei; Pan, Zhigeng; Li, Haisheng
• Is Part Of: Computers & graphics. Volume 70(2018)
• Page Start: 108
• Page End: 117
• Publication Date: 2018-02
• Subjects: 2D-3D alignment -- 3D Model retrieval -- ROI-based refinement -- 3D indoor scenes; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2017.07.029
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: Our approach produces CAD models of indoor objects from a single photo. We combine a deep CNN for object detection with a deformable part based alignment model. Users can interactively refine alignment results via shape similarities in user-specified ROIs. We use fuzzy correspondences between points on the 3D shapes to support ROI-based similarity searches. Graphical abstract: Abstract: Given a single indoor image, this paper proposes an automatic retrieval system to estimate the best-matching 3D models with consistent style and pose. To support this system, we combine a deep CNN based object detection approach with a deformable part based alignment model. The key idea is to cast a 2D-3D alignment problem as a part-based cross-domain matching. We also provide an interactive refinement interface that allows users to browse models based on similarities and differences between shapes in user-specified regions of interest (ROIs). We demonstrate the ability of our system on numerous examples.