Wrinkles Realistic Clothing Reconstruction by Combining Implicit and Explicit Method. (July 2023)

• Record Type: Journal Article
• Authors: Liu, Xinqi; Li, Jituo; Lu, Guodong
• Is Part Of: Computer aided design. Volume 160(2023)
• Publication Date: 2023-07
• Subjects: Clothing reconstruction -- Implicit and explicit combined method -- Realistic wrinkles -- A single image -- Learning and optimization combined method; Computer-aided design -- Periodicals; Engineering design -- Data processing -- Periodicals; Computer graphics -- Periodicals; Conception technique -- Informatique -- Périodiques; Infographie -- Périodiques; Computer graphics; Engineering design -- Data processing; Periodicals; Electronic journals; 620.00420285
• DOI: 10.1016/j.cad.2023.103514
• Languages: English
• ISSNs: 0010-4485

Abstract

Abstract: We propose a novel method to achieve robust and wrinkle-realistic clothing reconstruction from a single RGB image. Our approach successfully exploits both advantages of implicit function methods and explicit clothing generative models. The former allows to capture high-frequency wrinkle details and clothing appearance from a single image, and the latter provides reasonable clothing shape prior and structured topology. To utilize the implicit function method to obtain implicit clothing, the key is that we need to automatically segment the clothing region from the reconstructed results and solve its depth ambiguity problem. Therefore, we design a pixel-aligned segmentation method to achieve automatic and complete clothing segmentation. It relies on a lightweight clothing mask network and a simple but effective segmentation strategy. To deal with the depth ambiguity problem, a depth correction function is introduced to significantly remove the uncertainty in the depth direction and recover the correct clothing shape. Further, we utilize explicit clothing generative models to provide reasonable shapes and topologies. It first infers an initial explicit clothing template based on generative models. Then, the implicit clothing is fitted by this explicit template to capture high-frequency deformation, resulting in wrinkle-realistic and structured clothing results. Extensive experiments have proved the validity of our approach with the visual effect and geometry accuracy both reaching the state-of-the-art level in clothing reconstruction from a single image. Graphical abstract: Highlights: A novel clothing reconstruction method captures wrinkles realistic clothing results from a single image. A clothing segmentation method and depth correction function segment implicit clothing and solve the depth ambiguity. An information-fused clothing fitting method transfer the high-frequency details from implicit clothing to template clothing. Extensive experiments have proved that our method has reached the mainstream level in single-view clothing reconstruction.