A unified model for egocentric video summarization: an instance-based approach. (June 2021)

• Record Type: Journal Article
• Authors: Sreeja, M.U.; Kovoor, Binsu C.
• Is Part Of: Computers & electrical engineering. Volume 92(2021)
• Publication Date: 2021-06
• Subjects: Egocentric videos -- Ontology -- Multi-video summarization -- Single video summarization -- Object detection; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2021.107161
• Languages: English
• ISSNs: 0045-7906

Abstract

Highlights: A unique combined generic and query-based egocentric video summarization model. Addresses multi-video summarization as well based on deep learning and ontologies. Discrete custom trained instance based object and image detection models. Two novel datasets for experimentation in the respective egocentric video instances. An adaptive workflow for summarizing videos from any egocentric instance. Abstract: Video summarization generates compact representations of videos in the form of summaries. The proposed framework is a unified model for instance-driven egocentric video summarization addressing generic and query-based summarization along with multi-video summarization. The model employs deep learning for object detection and semantic web technologies in the form of ontologies for query inferences. Combining user preferences in the form of object queries has aided in producing summaries that are subjective in nature. Quantitative evaluations performed on two novel datasets namely, 'vehicle expo' and 'academic inspection' prove that the proposed framework produces remarkable results with the employment of instance-driven modules for summarization. Additional experimental analysis for shot boundary detection have been conducted based on proposed method and conventional methods establishing the significance of the instance-based model. Moreover, qualitative evaluations further ensure that the summaries are concise, representative, diverse and semantically relevant further substantiating the need for instance-driven models in video summarization. Graphical abstract: Image, graphical abstract