A decision support system for Rey–Osterrieth complex figure evaluation. (1st March 2023)

• Record Type: Journal Article
• Authors: Di Febbo, Davide; Ferrante, Simona; Baratta, Marco; Luperto, Matteo; Abbate, Carlo; Trimarchi, Pietro Davide; Giunco, Fabrizio; Matteucci, Matteo
• Is Part Of: Expert systems with applications. Volume 213:Part C(2023)
• Publication Date: 2023-03-01
• Subjects: Cognitive decline -- Rey Osterrieth complex figure -- Expert systems -- eHealth -- Computer vision -- Deep learning; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2022.119226
• Languages: English
• ISSNs: 0957-4174

Abstract

Abstract: Objective: The Rey Osterrieth complex figure (ROCF) is one of the most used neuropsychological tests for the assessment of mild cognitive impairment (MCI) and dementia. In the copy test, the patient has to draw a replica of a 18-pattern image and the outcome is a score based on the accuracy of the overall drawing. The standard scoring system however have limitations related to its subjective nature and its inability to evaluate other cognitive domains than constructional abilities. Previous works addressed those problems by proposing tablet-based automated evaluation systems. Even promising, such methods are still far away from clinical validation and translation. In this work, we developed a decision support system (DSS) for the evaluation of the ROCF copy test in the common practice using retrospective information from previously performed drawings. The goal of our system was to support the professionals providing a qualitative judgement for each of the 18 patterns, estimating the most probable diagnosis for the patient, and identifying the main signs associated to the obtained diagnosis. Methods: A total of 250 human evaluated ROCF copies were scanned from 57 healthy subjects, 131 individuals with MCI, and 62 individuals with dementia. The images were pre-processed and analysed using both computer vision and deep learning techniques to assign a qualitative label to the 18 patterns. Then, the 18 labels were used as features in 3 binary (healthy VS MCI, healthy VS dementia, MCI VS dementia) and a 3-class classifications with model explanation (SHAP). Results: Very good to excellent performance were obtained in all the diagnosis classification tasks. Indeed, an accuracy of about 85%, 91%, and 83% was obtained in discriminating healthy subjects from MCI, healthy subjects from dementia and MCI from dementia respectively. An accuracy of 73% was achieved in the 3-class classification. The model explanation showed which patterns are responsible for each prediction and how the importance of some patterns changes according to the severity of the cognitive decline. Significance: The proposed DSS enriches the standard evaluation and interpretation of the ROCF copy test. Being trained with retrospective knowledge, the performance of the DSS can be further enhanced by extending the dataset with existing ROCF copies. Highlights: A DSS can optimally discriminate healthy, MCI and dementia from the ROCF copy test. Explainable machine-learning methods support the interpretation of the ROCF result. Computer vision and deep learning can support pattern analysis and interpretation. Retrospective data is the appropriate data set to further improve the DSS performance.