Deep point-to-subspace metric learning for sketch-based 3D shape retrieval. (December 2019)

• Record Type: Journal Article
• Authors: Lei, Yinjie; Zhou, Ziqin; Zhang, Pingping; Guo, Yulan; Ma, Zijun; Liu, Lingqiao
• Is Part Of: Pattern recognition. Volume 96(2019:Dec.)
• Publication Date: 2019-12
• Subjects: Sketch-based 3D shape retrieval -- Cross-modality discrepancy -- Representative-view selection -- Point-to-subspace distance; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.106981
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: A representative-view selection (RVS) module is designed to identify the most representative views of a 3D shape for reducing the redundancy. A deep point-to-subspace metric learning (DPSML) module is proposed to calculate the query-adaptive similarity for sketch-based 3D shape retrieval. The representation learning problem is formulated as a classification problem with a specially designed classifier and training loss. State-of-the-art performance on SHREC 2013, 2014 and 2016 benchmarks are achieved. Abstract: One key issue in managing a large scale 3D shape dataset is to identify an effective way to retrieve a shape-of-interest. The sketch-based query, which enjoys the flexibility in representing the user's intention, has received growing interests in recent years due to the popularization of the touchscreen technology. Essentially, the sketch depicts an abstraction of a shape in a certain view while the shape contains the full 3D information. Matching between them is a cross-modality retrieval problem, and the state-of-the-art solution is to project the sketch and the 3D shape into a common space with which the cross-modality similarity can be calculated by the feature similarity/distance within. However, for a given query, only part of the viewpoints of the 3D shape is representative. Thus, blindly projecting a 3D shape into a feature vector without considering what is the query will inevitably bring query-unrepresentative information. To handle this issue, in this work we propose a Deep Point-to-Subspace Metric Learning (DPSML) framework to project a sketch into a feature vector and a 3D shape into a subspace spanned by a few selected basis feature vectors. The similarity between them is defined as the distance between the query feature vector and its closest point in the subspace by solving an optimization problem on the fly. Note that, the closest point is query-adaptive and can reflect the viewpoint information that is representative to the given query. To efficiently learn such a deep model, we formulate it as a classification problem with a special classifier design. To reduce the redundancy of 3D shapes, we also introduce a Representative-View Selection (RVS) module to select the most representative views of a 3D shape. By conducting extensive experiments on various datasets, we show that the proposed method can achieve superior performance over its competitive baseline methods and attain the state-of-the-art performance.