Searching part-specific neural fabrics for human pose estimation. (August 2022)

• Record Type: Journal Article
• Authors: Yang, Sen; Yang, Wankou; Cui, Zhen
• Is Part Of: Pattern recognition. Volume 128(2022)
• Publication Date: 2022-08
• Subjects: Human pose estimation -- Neural architecture search -- Cell-based neural fabrics -- Micro and macro search space -- Prior knowledge -- Part-specific; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2022.108652
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: We propose a novel micro and macro gradient-based architecture search space: parameterized cell-based neural fabric (CNF). With simple prior knowledge as guidance, our method automatically searches part-specific neural architectures to localize disentangled body parts, which extends the traditional part-based methods. Such part-specific neural architecture search can be seen as a variant of multi-task learning. It is a novel solution as multi-task neural architecture search for human pose estimation. The experiments show that the proposed model outperforms a hand-crafted part-based baseline model, and the resulting multiple part-specific architectures gain significant performance improvement against a single NAS-based architecture for the whole-body representation. Abstract: Neural architecture search (NAS) has emerged in many domains to jointly learn the architectures and weights of neural networks. The core spirit behind NAS is to automatically search neural architectures for target tasks with better performance-efficiency trade-offs. However, existing approaches emphasize on only searching a single architecture with less human intervention to replace a human-designed neural network, yet making the search process almost independent of the domain knowledge. In this paper, we aim to apply NAS for human pose estimation and we ask: when NAS meets this localization task, can the articulated human body structure help to search better task-specific architectures? To this end, we first design a new neural architecture search space, Cell-based Neural Fabric (CNF), to learn micro as well as macro neural architecture using a differentiable search strategy. Then, by viewing locating human parts as multiple disentangled prediction sub-tasks, we exploit the compositionality of human body structure as guidance to search multiple part-specific CNFs specialized for different human parts. After the search, all these part-specific neural fabrics have been tailored with distinct micro and macro architecture parameters. The results show that such knowledge-guided NAS-based model outperforms a hand-crafted part-based baseline model, and the resulting multiple part-specific architectures gain significant performance improvement against a single NAS-based architecture for the whole body. The experiments on MPII and COCO datasets show that our models 1 achieve comparable performance against the state-of-the-art methods while being relatively lightweight.