View-invariant action recognition via Unsupervised AttentioN Transfer (UANT). (May 2021)

• Record Type: Journal Article
• Authors: Ji, Yanli; Yang, Yang; Shen, Heng Tao; Harada, Tatsuya
• Is Part Of: Pattern recognition. Volume 113(2021)
• Publication Date: 2021-05
• Subjects: View-invariant recognition -- Cross-view evaluation -- Attention learning -- Transfer learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2020.107807
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose an Unsupervised AttentioN Transfer (UANT) network, that transfers attention from reference to arbitrary views to overcome view change.The UANT network learns a common attention map between different views via attention transfer. We exhaustively evaluate our approach on the UESTC and NTU dataset, and achieves outstanding recognition performance.Our proposed approach performs attention transfer in an unsupervised way, where view information is hidden. Abstract: With wide applications in surveillance and human-robot interaction, view-invariant human action recognition is critical, however, challenging, due to the action occlusion and information loss caused by view change. Current methods mainly seek for a common feature space for different views. However, such solutions become invalid when there exist few common features, e.g. large view change. To tackle the problem, we propose an Unsupervised AttentioN Transfer (UANT) approach for view-invariant action recognition. Other than transferring feature knowledge, UANT transfers attention from one selected reference view to arbitrary views, which correctly emphasizes crucial body joints and their relations for view-invariant representation. In addition, the attention calculation method taking into account both recognition contribution and reliability of skeleton joints generates effective attention. Experiments showed its effectiveness for correctly locating crucial body joints in action sequences. We exhaustively evaluate our approach on the UESTC and the NTU dataset, performing unsupervised view-invariant evaluations, i.e. X-view and Arbitrary-view recognition. Experiment results demonstrate its superiority in view-invariant representation and recognition.