Class‐aware single image to 3D object translational autoencoder. Issue 13 (14th October 2020)

• Record Type: Journal Article
• Authors: Turhan, Ceren Guzel; Bilge, Hasan Sakir
• Is Part Of: IET image processing. Volume 14:Issue 13(2020)
• Page Start: 3046
• Page End: 3053
• Publication Date: 2020-10-14
• Subjects: image segmentation -- feature extraction -- image reconstruction -- image coding; class‐aware single image -- 3D object translational autoencoder -- three‐dimensional domain -- reconstruction problem -- middle‐level features -- skip connections -- 2D features -- category‐agnostic model -- class‐annotations -- 3D reconstruction models -- intersection‐over‐union score -- object segmentation model -- reconstruction performance -- skipped volumetric class‐aware AE -- class‐aware nature; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2019.1152
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : The performances of generative adversarial network (GAN) and autoencoder (AE) models on images have been gathering a great deal of interest in terms of transferring them to three‐dimensional (3D) domain. In this study, single image to object reconstruction problem was focused by presenting a novel 2D‐to‐3D AE model inspired by the recent improvements. To benefit from middle‐level features, a model with skip connections was constructed by transferring 2D features to 3D domain. Moreover, the authors considered class‐awareness for obtaining a category‐agnostic model using limited class‐annotations. Apart from recent 3D reconstruction models, they adapted the intersection‐over‐union score based objective, which is used in the object segmentation model, for improving reconstruction performance. With all these contributions, they call their model as skipped volumetric class‐aware AE (SkipVCAE). According to experimental studies, proposed model obtained higher scores than the state‐of‐the‐art model given. The results have proven its performance as a category‐specific and category‐agnostic model together owing to its class‐aware nature. In the further analysis, it was seen that presented model yielded satisfactory results on a single image to object modelling compared to its multi‐view version thanks to class‐awareness.