A novel temporal moment retrieval model for apron surveillance video. (April 2023)

• Record Type: Journal Article
• Authors: Lyu, Zonglei; Zhang, Yilin
• Is Part Of: Computers & electrical engineering. Volume 107(2023)
• Publication Date: 2023-04
• Subjects: Video retrieval -- Temporal moment retrieval -- Object detection -- Heuristic rule; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2023.108616
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: This study addresses the problem of retrieving a specific moment from an untrimmed apron surveillance video through a query sentence. Existing methods cannot tackle it effectively by using sliding windows over the entire video, which suffers from exhaustively enumerated candidates. Here temporal and spatial characters of events are abstracted to corresponding heuristic rules in first-order predicate logic. Temporal characters are exploited to skip unimportant frames while potential time boundaries are deduced from spatial characters. Meanwhile, an object-detection model is designed to obtain spatial characters of potential time boundaries, which is compared with pre-defined heuristic rules for detection. If the obtained spatial characters do not match the heuristic rules, a multiple-time-scales searching module is devised to localize the boundaries. This method is evaluated on TACoS and Apron Surveillance Video (ASV) dataset. Results show that it outperforms the state-of-the-art methods on TACoS and accurately retrieves target events on ASV in one minute.