Cluster-wise learning network for multi-person pose estimation. (February 2020)

• Record Type: Journal Article
• Authors: Zhao, Ying; Luo, Zhiwei; Quan, Changqin; Liu, Dianchao; Wang, Gang
• Is Part Of: Pattern recognition. Volume 98(2020:Feb.)
• Publication Date: 2020-02
• Subjects: Pose estimation -- Keypoint detection -- Deep learning; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2019.107074
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: Cluster-wise keypoint detection. Instead of detect each keypoint separately, our network predicts multi-peak heatmaps for clusters of dense and sparse keypoints, which exploits global and local contextual information to improve the detection robustness. Feature aggregation. To enhance feature passing from shallow stack to deep stack, we aggregate information from different branches. The in-branch aggregation enriches the detection features in each branch by absorbing the holistic human region attention. The cross-branch aggregation further strengthens the detection features by fusing global and local context information between dense and sparse branches. Cluster-wise tag embedding. To better grouping the detected keypoints into instances, our network embeds relationships among the intra-cluster and inter-cluster keypoints with offset learning, which not only benefits the instance grouping but also individual keypoint identification. Graphical abstract: Abstract: In this paper, we propose a cluster-wise feature aggregation network that exploits multi-level contextual association for multi-person pose estimation. The recent popular approach for pose estimation is extracting the local maximum response from each detection heatmap that trained for a specific keypoint type. To exploit more contextual information, our network simultaneously learns complementary semantic information to encourage the detected keypoints subject to a certain contextual constraint. Specifically, our network uses dense and sparse branches to generate paired multi-peak detection heatmaps for clusters of keypoints. To enhance the feature passing through the network, we aggregate information from different branches. The in-branch aggregation enriches the detection features in each branch by absorbing the holistic human region attention. The cross-branch aggregation further strengthens the detection features by fusing global and local context information between dense and sparse branches. We demonstrate competitive performance of our network on the benchmark dataset for multi-person pose estimation.