P.140 Development and predictive validation of an intelligent surgical bimanual skills continuous assessment system. (June 2022)

• Record Type: Journal Article
• Authors: Yilmaz, R; Winkler-Schwartz, A; Mirchi, N; Reich, A; Ledwos, N; Del Maestro, R
• Is Part Of: Canadian journal of neurological sciences. Volume 49(2022)Supplement 1
• Page Start: S44
• Page End: S44
• Publication Date: 2022-06
• Subjects: Neurology -- Periodicals; Nervous system -- Surgery -- Periodicals; Electronic journals; 616.8
• DOI: 10.1017/cjn.2022.224
• Languages: English
• ISSNs: 0317-1671

Abstract

Abstract : Background: Surgeons' bimanual dexterity may correlate with the surgical outcome. Continuous assessment of psychomotor performance enables action-oriented feedback and error avoidance guidance. We outline an artificial intelligence (AI) application to continuously assess surgical bimanual skills and its predictive validation on surgical trainee performance throughout a neurosurgery residency program. Methods: Participants (n=50, 14 experts/neurosurgeons, 14 senior residents, 10 junior residents, 12 novices/medical students) performed two simulated subpial tumour resections a total of 300 times. A deep neural network was developed using expert/neurosurgeon and novice/medical student data to score bimanual performance at 0.2-second intervals between a score of 1.00 and -1.00. An average score was calculated for each task. Results: The average performance score differentiated among four expertise levels, p<.001. Neurosurgeons scored significantly higher than senior residents (p=.045) and junior residents scored significantly higher than medical students (p=.04). The intelligent system also differentiated between senior and junior trainee levels (p=.004). The performance score linearly correlated with resident year of neurosurgical training (adjusted R2=27.7%). Conclusions: The AI-powered intelligent system outlined is the first expert surgeon-data-based technical skills continuous assessment system, with predictive validity throughout a neurosurgical residency program. Intelligent systems may aid in the competency-based approach in surgery by accurately assessing trainee skills.