Temporal filtering networks for online action detection. (March 2021)

• Record Type: Journal Article
• Authors: Eun, Hyunjun; Moon, Jinyoung; Park, Jongyoul; Jung, Chanho; Kim, Changick
• Is Part Of: Pattern recognition. Volume 111(2021)
• Publication Date: 2021-03
• Subjects: Online action detection -- Temporal filtering networks -- Filter modules -- TFN; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107695
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a new approach, named Temporal Filtering Network (TFN), to boost the performance of online action detection by distinguishing between relevant and irrelevant temporal information. We simply implement TFN by introducing a filtering module without complex architectures. We perform extensive experiments on two benchmark datasets, where our TFN outperforms state-of-the-art methods by a large margin. We demonstrate the effectiveness and potential of our filtering modules by conducting comprehensive ablation studies. Abstract: Online action detection aims to detect a current action from an untrimmed, streaming video, where only current and past frames are available. Recent methods for online action detection have focused on how to model discriminative representations from temporally partial information. However, they overlook the fact that the input video contains background as well as actions. To overcome this problem, in this paper, we propose a novel approach, named Temporal Filtering Network, to distinguish between relevant and irrelevant information from a partially observed, untrimmed video. Specifically, we present a filtering module to learn relevance scores indicating how relevant the information is to a current action. Our filtering module emphasizes the relevant information to a current action, while it filters out the information of background and unrelated actions. We conduct extensive experiments on THUMOS-14 and TVSeries datasets. On these datasets, the proposed method outperforms state-of-the-art methods by a large margin. We also show the effectiveness of the filtering module through comprehensive ablation studies.