Web image annotation based on Tri-relational Graph and semantic context analysis. (May 2019)

• Record Type: Journal Article
• Authors: Zhang, Jing; Tao, Ti; Mu, Yakun; Sun, Han; Li, Dongdong; Wang, Zhe
• Is Part Of: Engineering applications of artificial intelligence. Volume 81(2019)
• Page Start: 313
• Page End: 322
• Publication Date: 2019-05
• Subjects: Tri-relational graph -- Semantic context analysis -- Multilevel random walk with restart -- Image label expansion; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2019.02.018
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: Web image annotation has became a hot research topic owing to massive image data and abundant semantic context. In this paper, we propose a Tri-relational Graph (TG) model for web image annotation, which comprises the image data graph, the region data graph and the label graph as subgraphs, and connects them by an additional tripartite graph induced from image segmentation results and label assignments. Through analyzing the global visual similarity between images, the visual similarity between regions, the semantic correlations between labels and the relationships between the three subgraphs by TG model, we perform multilevel Random Walk with Restart algorithm on TG to produce vertex-to-vertex relevance, including image-to-region, region-to-label and image-to-label relevances. Then semi-supervised learning is used to predict labels for unannotated image regions by inserting unlabeled images and their regions into TG. In addition, we also analyze the text context information of web image and achieve the semantic and proper nouns for the further label expansion through WordNet. Experiments on public web images datasets demonstrate that our proposed TG model and multilevel RWR algorithm can achieve good performance on image region annotation and outperform the similar image annotation methods. Moreover label expansion by web semantic context analysis can achieve more accurate and abundant annotation results.