PRAXIS: Towards automatic cognitive assessment using gesture recognition. (15th September 2018)

• Record Type: Journal Article
• Authors: Negin, Farhood; Rodriguez, Pau; Koperski, Michal; Kerboua, Adlen; Gonzàlez, Jordi; Bourgeois, Jeremy; Chapoulie, Emmanuelle; Robert, Philippe; Bremond, Francois
• Is Part Of: Expert systems with applications. Volume 106(2018)
• Page Start: 21
• Page End: 35
• Publication Date: 2018-09-15
• Subjects: Human computer interaction -- Computer assisted diagnosis -- Cybercare industry applications -- Medical services -- Patient monitoring -- Pattern recognition; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2018.03.063
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract : A computer-assisted cognitive assessment method based on Praxis test is proposed. Four methods are developed to evaluate dynamic and static gestures in Praxis test. A challenging RGB-D dataset is collected consisting of 60 subjects and 29 gestures. The deep learning based method outperformed the other approaches. The obtained results ensure that automatic cognitive assessment is feasible. Abstract: Praxis test is a gesture-based diagnostic test which has been accepted as diagnostically indicative of cortical pathologies such as Alzheimer's disease. Despite being simple, this test is oftentimes skipped by the clinicians. In this paper, we propose a novel framework to investigate the potential of static and dynamic upper-body gestures based on the Praxis test and their potential in a medical framework to automatize the test procedures for computer-assisted cognitive assessment of older adults. In order to carry out gesture recognition as well as correctness assessment of the performances we have recollected a novel challenging RGB-D gesture video dataset recorded by Kinect v2, which contains 29 specific gestures suggested by clinicians and recorded from both experts and patients performing the gesture set. Moreover, we propose a framework to learn the dynamics of upper-body gestures, considering the videos as sequences of short-term clips of gestures. Our approach first uses body part detection to extract image patches surrounding the hands and then, by means of a fine-tuned convolutional neural network (CNN) model, it learns deep hand features which are then linked to a long short-term memory to capture the temporal dependencies between video frames. We report the results of four developed methods using different modalities. The experiments show effectiveness of our deep learning based approach in gesture recognition and performance assessment tasks. Satisfaction of clinicians from the assessment reports indicates the impact of framework corresponding to the diagnosis.