An intelligent annotation-based image retrieval system based on RDF descriptions. (February 2017)

• Record Type: Journal Article
• Authors: Chen, Hua; Trouve, Antoine; Murakami, Kazuaki J; Fukuda, Akira
• Is Part Of: Computers & electrical engineering. Volume 58(2017)
• Page Start: 537
• Page End: 550
• Publication Date: 2017-02
• Subjects: Image retrieval -- Image annotation -- Semantic image retrieval -- RDF; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2016.09.031
• Languages: English
• ISSNs: 0045-7906

Abstract

Highlights: The notions of concept and instance are proposed to express the semantics of images. An image annotation model is proposed to annotate images at three levels. An intelligent ABIR system is implemented based on RDF descriptions. The problems of synonyms and homonyms are addressed in the our ABIR system. The proposed ABIR system provides a way to search with calculation. Graphical abstract: Abstract: In this paper, we aim at improving text-based image search using Semantic Web technologies. We introduce our notions of concept and instance in order to better express the semantics of images, and present an intelligent annotation-based image retrieval system. We test our approach on the Flickr8k dataset. From the provided captions, we generate annotations at three levels (sentence, concept and instance). These annotations are stored as RDF triples and can be queried to find images. The experimental results show that using concepts and instances to annotate images flexibly can improve the intelligence of the image retrieval system: (1) with annotations at concept level, it enables to create semantic links between concepts and then addresses many challenges, such as the problems of synonyms and homonyms; (2) with annotations at instance level, it can count things (e.g., "two people", "three animals") or identify a same concept.