Deep generation of 3D articulated models and animations from 2D stick figures. (December 2022)

• Record Type: Journal Article
• Authors: Akman, Alican; Sahillioğlu, Yusuf; Sezgin, T. Metin
• Is Part Of: Computers & graphics. Volume 109(2022)
• Page Start: 65
• Page End: 74
• Publication Date: 2022-12
• Subjects: Computer graphics -- 3D model generation -- Deep learning -- Sketch-based shape modeling; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2022.10.004
• Languages: English
• ISSNs: 0097-8493

Abstract

Abstract: Generating 3D models from 2D images or sketches is a widely studied important problem in computer graphics. We describe the first method to generate a 3D human model from a single sketched stick figure. In contrast to the existing human modeling techniques, our method does not require a statistical body shape model. We exploit Variational Autoencoders to develop a novel framework capable of transitioning from a simple 2D stick figure sketch, to a corresponding 3D human model. Our network learns the mapping between the input sketch and the output 3D model. Furthermore, our model learns the embedding space around these models. We demonstrate that our network can generate not only 3D models, but also 3D animations through interpolation and extrapolation in the learned embedding space. In addition to 3D human models, we produce 3D horse models in order to show the generalization ability of our framework. Extensive experiments show that our model learns to generate compatible 3D models and animations with 2D sketches. Graphical abstract: Highlights: Our method generates 3D models and animations from given 2D stick figure sketches. We exploit Variational Autoencoders in our framework for 2D sketch-3D model mapping. Our method does not require any other input such as a rigged template model.