Gesture Recognition in Augmented Reality Assisted Assembly Training. Issue 3 (March 2019)

• Record Type: Journal Article
• Authors: Dong, Jiaqi; Tang, Zisheng; Zhao, Qunfei
• Is Part Of: Journal of physics. Volume 1176:Issue 3(2019)
• Publication Date: 2019-03
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/1176/3/032030
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: The Augmented Reality assisted assembly training (ARAAT) is an effective and low-cost way in motor and electronic industry. In ARAAT, assembly operations are the processes which mainly use the AR device to recognize the gestures and match the virtual workpieces to the hand based on the consistency of time and space. Operations are recorded to the video by an AR device. By dealing with frames of the input video, operations can be distinguished. In this paper, a gesture recognition algorithm in ARAAT are proposed. The assembly training operations consist of several actions and actions we concerned are all conducted by gestures. Operations can be classified from the sequences of actions, that is, gestures. Every 20 frames of the input video are denoted as an action unit and the action unit slides with a time window. According to the 2D and 3D features of the action unit, a scorer trained by the samples of specific actions gives the optimal label of each frame to recognize the action. To avoid the disturbance of the transition and invalid action to the action boundary during the recognition, the boundary is iteratively optimized by the probability density distribution. The proposed algorithm implemented on HoloLens is compared with other algorithms. The experimental results indicate that the proposed algorithm achieves a high recognition rate and can reduce the computational complexity. The results prove the efficiency of recognition algorithm in ARAAT and ensure a friendly experience for the human-machine interaction.