Skeleton‐based attention‐aware spatial–temporal model for action detection and recognition. Issue 5 (7th April 2020)

• Record Type: Journal Article
• Authors: Cui, Ran; Zhu, Aichun; Wu, Jingran; Hua, Gang
• Is Part Of: IET computer vision. Volume 14:Issue 5(2020)
• Page Start: 177
• Page End: 184
• Publication Date: 2020-04-07
• Subjects: recurrent neural nets -- image motion analysis -- computer vision -- image representation -- image recognition -- video signal processing; human skeleton -- skeleton-based action analysis model -- action features -- static features -- dynamic features -- skeleton joints -- action location -- action recognition -- triple loss models action -- skeleton-based attention-aware spatial–temporal model -- action detection -- recurrent neural network framework -- conditional random field loss function; Computer vision -- Periodicals; Pattern recognition systems -- Periodicals; 006.37
• DOI: 10.1049/iet-cvi.2019.0751
• Languages: English
• ISSNs: 1751-9632

Abstract

Abstract : Action detection and recognition are popular subjects of research in the field of computer vision. The task of action detection can be regarded as the sum of action location and recognition. Action features described by using information concerning the human skeleton have the advantages of robustness against external factors and requiring a small amount of calculation. This study proposes a skeleton‐based action analysis model based on a recurrent neural network framework. The model learns action features by modelling static and dynamic features of skeleton joints and the importance of different video frames by introducing an attention module. For action location, conditional random field loss function is introduced to establish the context dependency of output labels. In the aspect of action recognition, the hierarchical training mechanism with triple loss models action features at coarse‐grained and fine‐grained levels. The authors' proposed method delivers state‐of‐the‐art results on action location and recognition tasks.