Video summarisation with visual and semantic cues. Issue 13 (2nd October 2020)

• Record Type: Journal Article
• Authors: Xu, Binwei; Liang, Haoran; Liang, Ronghua
• Is Part Of: IET image processing. Volume 14:Issue 13(2020)
• Page Start: 3134
• Page End: 3142
• Publication Date: 2020-10-02
• Subjects: video signal processing -- video retrieval -- image segmentation; semantic cues -- storage space -- good video summary -- human visual interestingness -- semantic level -- visual features -- visual cues -- dynamic video summarisation -- visual‐verbal saliency consistency -- semantic information -- important summary -- interesting summary -- high‐quality video summaries; Image processing -- Periodicals; 621.36705
• DOI: 10.1049/iet-ipr.2019.1355
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract : Video summarisation greatly improves the efficiency of people browsing videos and saves storage space. A good video summary should satisfy human visual interestingness and preserve the theme of the original video at the semantic level. Unlike many existing methods that consider only visual features to generate video summaries, this study proposes a method that combines visual and semantic cues to extract important information for dynamic video summarisation. The authors propose visual‐verbal saliency consistency to add semantic information and propose a novel attention motion, along with other visual features to fully represent visual interestingness. Based on the importance score of each frame calculated by combining these features, they select an optimal subset of segments to generate an important and interesting summary. They evaluate their method using the SumMe and TVSum datasets and experimental results show that their method generates high‐quality video summaries.