An attention-based deep learning approach for inertial motion recognition and estimation in human-robot collaboration. (April 2023)

• Record Type: Journal Article
• Authors: Zhou, Huiying; Yang, Geng; Wang, Baicun; Li, Xingyu; Wang, Ruohan; Huang, Xiaoyan; Wu, Haiteng; Wang, Xi Vincent
• Is Part Of: Journal of manufacturing systems. Volume 67(2023)
• Page Start: 97
• Page End: 110
• Publication Date: 2023-04
• Subjects: Human motion capture -- Human-centric smart manufacturing -- Motion tracking -- Human-robot collaboration -- Human-cyber-physical systems; Manufacturing processes -- Periodicals; Production engineering -- Data processing -- Periodicals; Robots, Industrial -- Periodicals; Production, Technique de la -- Informatique -- Périodiques; Robots industriels -- Périodiques; Electronic journals; 670.42
• DOI: 10.1016/j.jmsy.2023.01.007
• Languages: English
• ISSNs: 0278-6125

Abstract

Abstract: In line with a human-centric smart manufacturing vision, human-robot collaboration is striving to combine robots' high efficiency and quality with humans' rapid adaptability and high flexibility. In particular, perception, recognition and estimation of human motion determine when and what robot to collaborate with humans. This work presents an attention-based deep learning approach for inertial motion recognition and estimation in order to infer when robotic assistance will be requested by the human and to allow the robot to perform partial human tasks. First, in the stage of motion perception and recognition, quaternion-based calibration and forward kinematic analysis methods enable the reconstruction of human motion based on data streaming from an inertial motion capture device. Then, in the stage of motion estimation, residual module and Bidirectional Long Short-Term Memory module are integrated with proposed attention mechanism for estimating arm motion trajectories further. Experimental results show the effectiveness of the proposed approach in achieving better recognition and estimation in comparison with traditional approaches and existing deep learning approaches. It is experimentally verified in a laboratory environment involving a collaborative robot employed in a small part assembly task. Highlights: Inertial motion signals-based deep learning approach for motion recognition and estimation. Method allows acquiring and recognizing motion states using calibration and prior kinematics. Combination of attention mechanism and residual module improves trajectory estimation accuracy. A human-robot collaborative assembly case validates the effectiveness of proposed approach.