Classification of motor errors to provide real-time feedback for sports coaching in virtual reality — A case study in squats and Tai Chi pushes. (November 2018)

• Record Type: Journal Article
• Authors: Hülsmann, Felix; Göpfert, Jan Philip; Hammer, Barbara; Kopp, Stefan; Botsch, Mario
• Is Part Of: Computers & graphics. Volume 76(2018)
• Page Start: 47
• Page End: 59
• Publication Date: 2018-11
• Subjects: Sports coaching in virtual reality -- Motor learning environments -- Motor performance quality -- Human motion analysis -- Auto-generated augmented feedback; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2018.08.003
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: Classification of movement errors as basis for sports coaching in virtual reality. Basis for generation of verbal and augmented feedback in virtual environments. New online pipeline for classification and feedback generation. Two data sets (squats, Tai Chi movements) used as test case. Graphical abstract: Abstract: For successful fitness coaching in virtual reality, movements of a trainee must be analyzed in order to provide feedback. To date, most coaching systems only provide coarse information on movement quality. We propose a novel pipeline to detect a trainee's errors during exercise that is designed to automatically generate feedback for the trainee. Our pipeline consists of an online temporal warp of a trainee's motion, followed by Random-Forest-based feature selection. The selected features are used for the classification performed by Support Vector Machines. Our feedback to the trainee can consist of predefined verbal information as well as automatically generated visual augmentations. For the latter, we exploit information on feature importance to generate real-time feedback in terms of augmented color highlights on the trainee's avatar. We show our pipeline's superiority over two popular approaches from human activity recognition applied to our problem, k-Nearest Neighbor, combined with Dynamic Time Warping (KNN-DTW), as well as a recent combination of Convolutional Neural Networks with a Long Short-term Memory Network. We compare classification quality, time needed for classification, as well as the classifiers' ability to automatically generate augmented feedback. In an exemplary application, we demonstrate that our pipeline is suitable to deliver verbal as well as automatically generated augmented feedback inside a CAVE-based sports training environment in virtual reality.