Augmented reality-based surgical guidance for wrist arthroscopy with bone-shift compensation. (March 2023)

• Record Type: Journal Article
• Authors: Jeung, Deokgi; Jung, Kyunghwa; Lee, Hyun-Joo; Hong, Jaesung
• Is Part Of: Computer methods and programs in biomedicine. Volume 230(2023)
• Publication Date: 2023-03
• Subjects: Wrist arthroscopy -- Augmented reality -- Bone-shift compensation -- In vivo computed tomography; AR augmented reality -- CT computed tomography -- D-H Denavit–Hartenberg -- DTM dart-thrower's motion -- FE flexion-extension -- FOV field of view -- ICP iterative closest point -- IK inverse kinematics -- MR magnetic resonance -- OpenCV Open Source Computer Vision -- OTS optical tracking system -- PCA principal component analysis -- RUD radial-ulnar deviation -- VTK Visualization Toolkit; Medicine -- Computer programs -- Periodicals; Biology -- Computer programs -- Periodicals; Computers -- Periodicals; Medicine -- Periodicals; Médecine -- Logiciels -- Périodiques; Biologie -- Logiciels -- Périodiques; Biology -- Computer programs; Medicine -- Computer programs; Periodicals; Electronic journals; 610.28
• DOI: 10.1016/j.cmpb.2022.107323
• Languages: English
• ISSNs: 0169-2607

Abstract

Highlights: Real-time augmented reality-based surgical guidance to enable bone shift compensation during surgery. The shifted wrist bone is estimated in patient-specific virtual bone model by inverse kinematics based on noninvasive fiducial markers. Abstract: Background and objectives: Intraoperative joint condition is different from preoperative CT/MR due to the motion applied during surgery, inducing an inaccurate approach to surgical targets. This study aims to provide real-time augmented reality (AR)-based surgical guidance for wrist arthroscopy based on a bone-shift model through an in vivo computed tomography (CT) study. Methods: To accurately visualize concealed wrist bones on the intra-articular arthroscopic image, we propose a surgical guidance system with a novel bone-shift compensation method using noninvasive fiducial markers. First, to measure the effect of traction during surgery, two noninvasive fiducial markers were attached before surgery. In addition, two virtual link models connecting the wrist bones were implemented. When wrist traction occurs during the operation, the movement of the fiducial marker is measured, and bone-shift compensation is applied to move the virtual links in the direction of the traction. The proposed bone-shift compensation method was verified with the in vivo CT data of 10 participants. Finally, to introduce AR, camera calibration for the arthroscope parameters was performed, and a patient-specific template was used for registration between the patient and the wrist bone model. As a result, a virtual bone model with three-dimensional information could be accurately projected on a two-dimensional arthroscopic image plane. Results: The proposed method was possible to estimate the position of wrist bone in the traction state with an accuracy of 1.4 mm margin. After bone-shift compensation was applied, the target point error was reduced by 33.6% in lunate, 63.3% in capitate, 55.0% in scaphoid, and 74.8% in trapezoid than those in preoperative wrist CT. In addition, a phantom experiment was introduced simulating the real surgical environment. AR display allowed to expand the field of view (FOV) of the arthroscope and helped in visualizing the anatomical structures around the bones. Conclusions: This study demonstrated the successful handling of AR error caused by wrist traction using the proposed method. In addition, the method allowed accurate AR visualization of the concealed bones and expansion of the limited FOV of the arthroscope. The proposed bone-shift compensation can also be applied to other joints, such as the knees or shoulders, by representing their bone movements using corresponding virtual links. In addition, the movement of the joint skin during surgery can be measured using noninvasive fiducial markers in the same manner as that used for the wrist joint.