Automated outcome scoring in a virtual reality simulator for endodontic surgery. (January 2018)

• Record Type: Journal Article
• Authors: Yin, Myat Su; Haddawy, Peter; Suebnukarn, Siriwan; Rhienmora, Phattanapon
• Is Part Of: Computer methods and programs in biomedicine. Volume 153(2018)
• Page Start: 53
• Page End: 59
• Publication Date: 2018-01
• Subjects: Surgical simulation -- Virtual reality -- Automated outcome assessment -- Formative assessment -- Intelligent tutoring; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2017.10.001
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: Numerous VR simulators have been developed as a means of addressing limitations of the traditional apprenticeship approach to dental surgical skill training. Most existing simulators support intra- and extra-coronal procedures such as caries removal. We address the problem of automated outcome assessment for endodontic surgery. Outcome assessment is an essential component in providing formative feedback, which requires assessing the outcome, relating it to the procedure, and communicating in a language natural to dental students. This paper takes a first step toward automated generation of such comprehensive feedback. Detailed scores are transformed into the standard scoring language used by dental schools. Results from the evaluation of our system with expert endodontists show a high degree of agreement with expert scores. Abstract: Background and objective: We address the problem of automated outcome assessment in a virtual reality (VR) simulator for endodontic surgery. Outcome assessment is an essential component of any system that provides formative feedback, which requires assessing the outcome, relating it to the procedure, and communicating in a language natural to dental students. This study takes a first step toward automated generation of such comprehensive feedback. Methods: Virtual reference templates are computed based on tooth anatomy and the outcome is assessed with a 3D score cube volume which consists of voxel-level non-linear weighted scores based on the templates. The detailed scores are transformed into standard scoring language used by dental schools. The system was evaluated on fifteen outcome samples that contained optimal results and those with errors including perforation of the walls, floor, and both, as well as various combinations of major and minor over and under drilling errors. Five endodontists who had professional training and varying levels of experiences in root canal treatment participated as raters in the experiment. Results: Results from evaluation of our system with expert endodontists show a high degree of agreement with expert scores (information based measure of disagreement 0.04-0.21). At the same time they show some disagreement among human expert scores, reflecting the subjective nature of human outcome scoring. The discriminatory power of the AOS scores analyzed with three grade tiers (A, B, C) using the area under the receiver operating characteristic curve (AUC). The AUC values are generally highest for the {AB: C} cutoff which is cutoff at the boundary between clinically acceptable (B) and clinically unacceptable (C) grades. Conclusions: The objective consistency of computed scores and high degree of agreement with experts make the proposed system a promising addition to existing VR simulators. The translation of detailed level scores into terminology commonly used in dental surgery supports natural communication with students and instructors. With the reference virtual templates created automatically, the approach is robust and is applicable in scoring the outcome of any dental surgery procedure involving the act of drilling.