Augmented reality calibration using feature triangulation iteration-based registration for surgical navigation. (September 2022)

• Record Type: Journal Article
• Authors: Shao, Long; Yang, Shuo; Fu, Tianyu; Lin, Yucong; Geng, Haixiao; Ai, Danni; Fan, Jingfan; Song, Hong; Zhang, Tao; Yang, Jian
• Is Part Of: Computers in biology and medicine. Volume 148(2022)
• Publication Date: 2022-09
• Subjects: Augmented reality -- Surgical navigation -- Shape feature matching -- Feature triangulation iteration registration; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2022.105826
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Background: Marker-based augmented reality (AR) calibration methods for surgical navigation often require a second computed tomography scan of the patient, and their clinical application is limited due to high manufacturing costs and low accuracy. Methods: This work introduces a novel type of AR calibration framework that combines a Microsoft HoloLens device with a single camera registration module for surgical navigation. A camera is used to gather multi-view images of a patient for reconstruction in this framework. A shape feature matching-based search method is proposed to adjust the size of the reconstructed model. The double clustering-based 3D point cloud segmentation method and 3D line segment detection method are also proposed to extract the corner points of the image marker. The corner points are the registration data of the image marker. A feature triangulation iteration-based registration method is proposed to quickly and accurately calibrate the pose relationship between the image marker and the patient in the virtual and real space. The patient model after registration is wirelessly transmitted to the HoloLens device to display the AR scene. Results: The proposed approach was used to conduct accuracy verification experiments on the phantoms and volunteers, which were compared with six advanced AR calibration methods. The proposed method obtained average fusion errors of 0.70 ± 0.16 and 0.91 ± 0.13 mm in phantom and volunteer experiments, respectively. The fusion accuracy of the proposed method is the highest among all comparison methods. A volunteer liver puncture clinical simulation experiment was also conducted to show the clinical feasibility. Conclusions: Our experiments proved the effectiveness of the proposed AR calibration method, and revealed a considerable potential for improving surgical performance. Highlights: An augmented reality calibration framework enables accurate fusion. Scale correction of the reconstructed model improves the fusion quality. Accurate recognition of image marker corners improves the fusion accuracy. Accurate calibration of pose between marker and patient reduces fusion errors.