An NLP-guided ontology development and refinement approach to represent and query visual information. (1st March 2023)

• Record Type: Journal Article
• Authors: Patel, Ashish Singh; Merlino, Giovanni; Puliafito, Antonio; Vyas, Ranjana; Vyas, O.P.; Ojha, Muneendra; Tiwari, Vivek
• Is Part Of: Expert systems with applications. Volume 213:Part B(2023)
• Publication Date: 2023-03-01
• Subjects: Semantic web -- Multimedia representation -- Ontology engineering -- Knowledge graph -- Information retrieval; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.118998
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: The ubiquitous presence of surveillance systems generates massive amounts of video data. Storage and analysis of this data in real-time is a substantial challenge. There is huge potential in representing data in machine-readable and machine-interpretable format due to the presence of hidden semantics in images and videos. However, such representation requires ontology, which calls for expert domain knowledge. In this paper, a novel NLP-guided approach to generate an ontology for multimedia representation and information retrieval is proposed. A semi-automatic NLP-guided framework, which extracts all possible relations among objects is presented. This framework leverages the textual data of the domain to generate possible descriptions and actions within the domain. Relations among objects get embedded as object properties, whereas the category of an object as a class. Features and attributes of objects encode the data properties of the ontology. The proposed ontology is compared with existing multimedia ontologies and evaluated with regard to its capability to represent relations occurring in benchmark datasets, demonstrating the completeness and thorough coverage of the domain concepts. Spatial reasoning rules are established using Semantic Web Rule Language (SWRL) rules, and information retrieval is demonstrated using Description Logic (DL) and SPARQL queries. The proposed NLP-guided ontology generation approach is general enough to help in the development of ontologies for other domains as well, by providing video and textual data of the domain of interest, with limited human involvement. Highlights: Object properties extraction with textbase and NLP-guided approach. Frequency-based semantic refinement to enhance domain-specific relations. A novel data-driven video surveillance ontology "VizITS" for the ITS domain. Semantic information retrieval as events, metadata, and question-answering.