Image-based 3D model retrieval via disentangled feature learning and enhanced semantic alignment. Issue 2 (March 2023)

• Record Type: Journal Article
• Authors: Nie, Jie; Zhang, Ting; Li, Tianbao; Yu, Shusong; Li, Xuanya; Wei, Zhiqiang
• Is Part Of: Information processing & management. Volume 60:Issue 2(2023)
• Publication Date: 2023-03
• Subjects: 3D model retrieval -- Unsupervised domain adaptation -- Feature disentangle learning; Information storage and retrieval systems -- Periodicals; Information science -- Periodicals; Systèmes d'information -- Périodiques; Sciences de l'information -- Périodiques; Information science; Information storage and retrieval systems; Periodicals; 658.4038
• DOI: 10.1016/j.ipm.2022.103159
• Languages: English
• ISSNs: 0306-4573

Abstract

Abstract: With the development of 3D technology and the increase in 3D models, 2D image-based 3D model retrieval tasks have drawn increased attention from scholars. Previous works align cross-domain features via adversarial domain alignment and semantic alignment. However, the extracted features of previous methods are disturbed by the residual domain-specific features, and the lack of labels for 3D models makes the semantic alignment challenging. Therefore, we propose disentangled feature learning associated with enhanced semantic alignment to address these problems. On one hand, the disentangled feature learning enables decoupling the twisted raw features into the isolated domain-invariant and domain-specific features, and the domain-specific features will be dropped while performing adversarial domain alignment and semantic alignment to acquire domain-invariant features. On the other hand, we mine the semantic consistency by compacting each 3D model sample and its nearest neighbors to further enhance semantic alignment for unlabeled 3D model domain. We give comprehensive experiments on two public datasets, and the results demonstrate the superiority of the proposed method. Especially on MI3DOR-2 dataset, our method outperforms the current state-of-the-art methods with gains of 2.88% for the strictest retrieval metric NN. Highlights: An end-to-end unsupervised 2D image-based 3D model retrieval framework. Transferring knowledge from labeled 2D images to unlabeled 3D models. Domain-invariant features are disentangled from the original features. Nearest neighbors enhanced semantic alignment for cross domain feature learning. Experiments on MI3DOR and MI3DOR-2 verified the superiority of the method.