A Malleable Boundary Network for temporal action detection. (October 2022)

• Record Type: Journal Article
• Authors: Wang, Tian; Hou, Boyao; Li, Zexian; Li, Zhe; Huang, Lei; Zhang, Baochang; Snoussi, Hichem
• Is Part Of: Computers & electrical engineering. Volume 103(2022)
• Publication Date: 2022-10
• Subjects: Temporal action detection -- Temporal action proposal generation -- Untrimmed video -- MB-Net; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108250
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: Temporal action detection in untrimmed videos is a challenging task aiming to predict the boundary and category of action instances. It can be useful in transportation. In this study, we propose a two-stage framework Malleable Boundary Network (MB-Net) to adaptively regress proposals based on finer scores. In particular, MB-Net consists of a Potential Boundary Generator in the first stage and an Adaptive Proposal Detector in the second stage. First, the Potential Boundary Generator fuses multiple sets of flexible score sequences to obtain tentative proposals through a frame-level feature in an anchor-free way. Then, the Adaptive Proposal Detector employs parallel modules to filter, classify and regress proposals adaptively. Besides, we propose an easy-to-realize feature augmented method Structured Temporal Segment Pooling, which makes full use of the information throughout the whole proposal. Experiments show that MB-Net achieves state-of-the-art performance on popular benchmarks THUMOS-14 and Activity-1.3 with an improvement of 1.9% and 1.2%. Graphical abstract: Highlights: A two-stage MB-Net framework for temporal action detection. Treating detection process as coarse-grained construction and fine-grained adjustment. A boundary generator with large receptive fields and a well-designed fusion mechanism. A brief and parallel instance detector with a designed temporal pooling method.