IMMAT: Mesh Reconstruction from Single View Images by Medial Axis Transform Prediction. (September 2022)

• Record Type: Journal Article
• Authors: Hu, Jianwei; Chen, Gang; Yang, Baorong; Wang, Ningna; Guo, Xiaohu; Wang, Bin
• Is Part Of: Computer aided design. Volume 150(2022)
• Publication Date: 2022-09
• Subjects: Deep learning -- Medial Axis Transform -- 3D reconstruction -- Single view image; Computer-aided design -- Periodicals; Engineering design -- Data processing -- Periodicals; Computer graphics -- Periodicals; Conception technique -- Informatique -- Périodiques; Infographie -- Périodiques; Computer graphics; Engineering design -- Data processing; Periodicals; Electronic journals; 620.00420285
• DOI: 10.1016/j.cad.2022.103304
• Languages: English
• ISSNs: 0010-4485

Abstract

Abstract: The representation of a 3D shape is a key element for capturing the overall structure as well as the local details. In this paper, we propose to predict a mesh representation of the Medial Axis Transform (called medial mesh) as an intermediate representation with our IMMAT framework, to reconstruct the 3D shape from a single view image. Because the MAT contains the skeleton topology and local thickness information, it not only enhances the ability to reconstruct topologically complex shapes but also better preserves the local details with its thickness control. The framework consists of three modules. The Image2Sphere module predicts the medial spheres inside the shape surface and the Topology Prediction module predicts the topological relationship (skeleton) between the predicted spheres. Then the MAT Smoothing module smooths the predicted MAT and fine-tunes the coordinates and radii of the spheres by graph convolution. The final 3D surface can be reconstructed by converting the predicted MAT to an implicit surface through CSG operation and then extracting the boundary surface through Marching Cubes. Experimental results show that our method outperforms the state-of-the-art methods both quantitatively and qualitatively on the reconstruction task. Graphical abstract: Highlights: We utilize medial axis transform for 3D reconstruction from single view images. We propose a learning-based method to predict medial spheres from single view images. We predict the topology relationships of 3D spheres to form a medial axis transform.