Marker‐less real‐time intra‐operative camera and hand‐eye calibration procedure for surgical augmented reality. Issue 6 (12th November 2019)

• Record Type: Journal Article
• Authors: Kalia, Megha; Mathur, Prateek; Navab, Nassir; Salcudean, Septimiu E.
• Is Part Of: Healthcare technology letters. Volume 6:Issue 6(2019)
• Page Start: 255
• Page End: 260
• Publication Date: 2019-11-12
• Subjects: medical robotics -- medical image processing -- cameras -- endoscopes -- calibration -- robot vision -- surgery -- phantoms -- rendering (computer graphics) -- augmented reality -- image registration -- biomedical optical imaging; hand‐eye calibration procedure -- surgical augmented reality -- pre‐operative medical data -- camera intrinsic matrix estimation -- hand‐eye transformation -- camera calibration -- endoscope -- high visual error -- virtual rendered tool tip -- marker‐less real‐time intra‐operative camera -- augmented reality rendering -- subsequent gradient descent steps -- da Vinci robot -- average target registration error -- prostate phantom; Biomedical engineering -- Periodicals; Medical technology -- Periodicals; 610.28
• DOI: 10.1049/htl.2019.0094
• Languages: English
• ISSNs: 2053-3713

Abstract

Abstract : Accurate medical Augmented Reality (AR) rendering requires two calibrations, a camera intrinsic matrix estimation and a hand‐eye transformation. We present a unified, practical, marker‐less, real‐time system to estimate both these transformations during surgery. For camera calibration we perform calibrations at multiple distances from the endoscope, pre‐operatively, to parametrize the camera intrinsic matrix as a function of distance from the endoscope. Then, we retrieve the camera parameters intra‐operatively by estimating the distance of the surgical site from the endoscope in less than 1 s. Unlike in prior work, our method does not require the endoscope to be taken out of the patient; for the hand‐eye calibration, as opposed to conventional methods that require the identification of a marker, we make use of a rendered tool‐tip in 3D. As the surgeon moves the instrument and observes the offset between the actual and the rendered tool‐tip, they can select points of high visual error and manually bring the instrument tip to match the virtual rendered tool tip. To evaluate the hand‐eye calibration, 5 subjects carried out the hand‐eye calibration procedure on a da Vinci robot. Average Target Registration Error of approximately 7mm was achieved with just three data points.