A fusion-based spiking neural network approach for predicting collaboration request in human-robot collaboration. (December 2022)

• Record Type: Journal Article
• Authors: Zhang, Rong; Li, Jie; Zheng, Pai; Lu, Yuqian; Bao, Jinsong; Sun, Xuemin
• Is Part Of: Robotics and computer-integrated manufacturing. Volume 78(2022)
• Publication Date: 2022-12
• Subjects: Human-robot collaboration -- Spiking neural networks -- Collaboration prediction; Robots, Industrial -- Periodicals; Computer integrated manufacturing systems -- Periodicals; Robotics -- Periodicals; Robots industriels -- Périodiques; Productique -- Périodiques; Robotique -- Périodiques; 670.285
• DOI: 10.1016/j.rcim.2022.102383
• Languages: English
• ISSNs: 0736-5845

Abstract

Highlights: Compared with the traditional assembly method, a fusion-based spiking neural network approach for predicting collaboration request proposed in this paper has the following advantages. A human collaboration request prediction method under the joint influence of multi-channel information is proposed. The FSNN method for asynchronous monitoring is proposed to reduce the workload of non-time-series feature detection. The coding methods of different input information, and the threshold adaptive updating method under the influence of different neurons are proposed. Compared with different prediction methods, the superiority of the FSNN scheme is proved. Abstract: In human-robot collaborative (HRC) manufacturing systems, how the collaborative robots engage in the collaborative tasks and complete the corresponding work in a timely manner according to the actual state has been a critical factor that hinders the efficiency of HRC. Inappropriate collaborative behaviors will result in a poor perceptual experience for human operators (e.g., robots starting action too early or too late). To address this issue, a fusion-based spiking neural networks (FSNNs) approach for collaboration request prediction is proposed, aiming to find the right collaboration timing for robots in HRC assembly system and to minimize human aversion without affecting human operation behaviors. By encoding human behavior, product state and robot pose into spiking signals that can be processed by FSNNs, the spatio-temporal coupling relationship between those three aspects can be comprehensively analyzed, and to solve the appropriate timing of robot participation in collaboration. Finally, demonstrative experiments are carried out on the HRC assembly of generator end caps in the lab environment. Compared with the baseline methods, the decision accuracy of the proposed one is improved by nearly 30%, which further proves its effectiveness.