A deep learning method for video‐based action recognition. Issue 14 (7th July 2021)

• Record Type: Journal Article
• Authors: Zhang, Guanwen; Rao, Yukun; Wang, Changhao; Zhou, Wei; Ji, Xiangyang
• Is Part Of: IET image processing. Volume 15:Issue 14(2021)
• Page Start: 3498
• Page End: 3511
• Publication Date: 2021-07-07
• Subjects: Image processing -- Periodicals; 621.36705
• DOI: 10.1049/ipr2.12303
• Languages: English
• ISSNs: 1751-9659

Abstract

Abstract: In this paper, a deep learning method for video‐based action recognition is proposed. On the one hand, boundary compensation on the basis of a deep neural network is performed to achieve action proposal. Boundary compensation considering non‐maximum suppression according to sliding window priority is applied to remove redundant windows. To accurately detect boundaries, a boundary compensation network is established with multiple networks to process different numbers of segments. On the other hand, action recognition based on the resultant action proposals is performed. To further utilise boundary compensation, three methods are introduced for key frame selection. Optical flow and RGB features are combined via a channel fusion to realise feature representation. A two‐stream network with a spatiotemporal structure is adopted for action recognition. The proposed method is evaluated on three public datasets. The experimental results demonstrate that the proposed method achieves a superior performance to that of state‐of‐the‐art methods.