Video‐based action recognition using spurious‐3D residual attention networks. Issue 11 (26th May 2022)

• Record Type: Journal Article
• Authors: Chen, Bo; Tang, Hongying; Zhang, Zebin; Tong, Guanjun; Li, Baoqing
• Is Part Of: IET image processing. Volume 16:Issue 11(2022)
• Page Start: 3097
• Page End: 3111
• Publication Date: 2022-05-26
• Subjects: Image processing -- Periodicals; 621.36705
• DOI: 10.1049/ipr2.12541
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract: Recently, 3D Convolutional Neural Networks (3D CNNs) have attracted extensive attention in extracting spatial and temporal features in videos for their efficient feature extraction ability. However, it also brings enormous model parameters by training very deep 3D CNNs. Here, a novel network named spurious‐3D Residual Attention Networks (S3D RANs) is proposed for video‐based action recognition, which has the powerful capacity to learn collaborative spatiotemporal features. In particular, by leveraging the merits from 2D Convolutional Neural Networks (2D CNNs) and 3D CNNs, 2D CNNs are applied rather than 3D CNNs on frames of the single view of volumetric videos data to learn temporal motion features directly. Furthermore, view and channel‐wise attention mechanism submodules are employed in the residual unit to learn the importance of each view for action recognition and guide the network to pay more attention to the more useful information for action recognition. Experimental results on UCF‐101, HMDB‐51 datasets demonstrate that our S3D RANs have higher accuracy and lower model complexity than existing works.