Deep-seated features histogram: A novel image retrieval method. (August 2021)

• Record Type: Journal Article
• Authors: Liu, Guang-Hai; Yang, Jing-Yu
• Is Part Of: Pattern recognition. Volume 116(2021)
• Publication Date: 2021-08
• Subjects: Image retrieval -- VGG-16 network -- orientation selection -- color perception -- deep-seated features histogram; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2021.107926
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Low-level features are extracted by simulating the human orientation selection and color perception mechanisms. Ranking whitening is proposed for extracting deep features via low-level features and reasonably combining them to obtain deep-seated features. The proposed method is straightforward and reduces the vector dimensionality. Deep-seated features can describe image contents in terms of colors and edge orientations and identify similar scene styles. Abstract: Low-level features and deep features each have their own advantages and disadvantages in image representation. However, combining their advantages within a CBIR framework remains challenging. To address this problem, we propose a novel image-retrieval method: the deep-seated features histogram (DSFH). Its main highlights are: 1) Low-level features are extracted by simulating the human orientation selection and color perception mechanisms. This follows the human habit of looking at conspicuous regions and then less-conspicuous ones. 2) A novel method, ranking whitening, is proposed for extracting deep features via low-level features and combining them to obtain deep-seated features. 3) The proposed method is straightforward and reduces the vector dimensionality of the FC7 layer of a pre-trained VGG-16 network, and significantly improves image-retrieval precision. Comparative experiments demonstrate that the proposed method outperforms several state-of-the-art methods, including low-level feature-based, deep feature-based, and fused feature-based methods, in terms of precision/recall, area under the precision/recall curve metrics, and mean average precision. The proposed method provides efficient CBIR performance and not only has the power to discriminate low-level features, including color, texture, and shape, but can also match scenes of similar style.