Sketching out the details: Sketch-based image retrieval using convolutional neural networks with multi-stage regression. (April 2018)

• Record Type: Journal Article
• Authors: Bui, Tu; Ribeiro, Leonardo; Ponti, Moacir; Collomosse, John
• Is Part Of: Computers & graphics. Volume 71(2018)
• Page Start: 77
• Page End: 87
• Publication Date: 2018-04
• Subjects: Sketch based image retrieval (SBIR) -- Deep learning -- Cross-domain modeling -- Compact feature representations -- Multi-stage regression -- Contrastive and triplet losses; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2017.12.006
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: A generic multi-stage training methodology for cross-domain learning. An extensive evaluation of convnet architectures and weight sharing strategies. State-of- art performance on three standard SBIR benchmarks. Abstract: We propose and evaluate several deep network architectures for measuring the similarity between sketches and photographs, within the context of the sketch based image retrieval (SBIR) task. We study the ability of our networks to generalize across diverse object categories from limited training data, and explore in detail strategies for weight sharing, pre-processing, data augmentation and dimensionality reduction. In addition to a detailed comparative study of network configurations, we contribute by describing a hybrid multi-stage training network that exploits both contrastive and triplet networks to exceed state of the art performance on several SBIR benchmarks by a significant margin. Datasets and models are available athttp://www.cvssp.org . Graphical abstract: