Deep Multi-Similarity Hashing with semantic-aware preservation for multi-label image retrieval. (1st November 2022)

• Record Type: Journal Article
• Authors: Qin, Qibing; Xian, Lintao; Xie, Kezhen; Zhang, Wenfeng; Liu, Yu; Dai, Jiangyan; Wang, Chengduan
• Is Part Of: Expert systems with applications. Volume 205(2022)
• Publication Date: 2022-11-01
• Subjects: Deep hashing -- Image retrieval -- Higher-order features -- Semantic-aware similarity -- Multi-similarity loss; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.117674
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Recently, many deep supervised hashing has been developed for multi-label image retrieval applications and has already achieved good effects. However, current methods quantify the similarities between image pairs by counting the number of shared labels roughly, totally ignoring the semantic relationships of image pairs. Besides, the sample imbalance of different classes could bias the training process toward majority categories, deteriorating the model performance. To solve the above-mentioned problems, we propose a novel deep supervised hashing framework, named Deep Multi-Similarity Hashing with semantic-aware preserving (DMSH), to generate binary codes with high-level semantic similarity. Specifically, to obtain distinguishing features, Convolutional Neural Networks (CNN) with correlation operators are introduced to capture discriminative semantic features of the input images, including auto-correlation characteristics and cross-correlation characteristics. By jointly exploiting label-level and semantic-level similarity, the semantic-aware similarity module is developed to quantify the high-level semantic similarity of image pairs, distinguishing the differences beyond categories. Furthermore, the multi-similarity loss is incorporated into deep hashing to collect informative pairs efficiently, which could alleviate the impact of imbalanced sample pairs during model training. Extensive comparison experiments on three benchmark datasets demonstrate that our DMSH achieves promising performance with respect to different evaluation metrics. Highlights: CNNs with correlation operators are introduced to capture higher-order features. A semantic-aware method is proposed to quantify the similarity of image pairs. Multi-similarity loss is developed to collect informative pairs efficiently. The state-of-the-art results are reported on three benchmark datasets.