Optical biopsy mapping on endoscopic image mosaics with a marker-free probe. (April 2022)

• Record Type: Journal Article
• Authors: Zenteno, Omar; Trinh, Dinh-Hoan; Treuillet, Sylvie; Lucas, Yves; Bazin, Thomas; Lamarque, Dominique; Daul, Christian
• Is Part Of: Computers in biology and medicine. Volume 143(2022)
• Publication Date: 2022-04
• Subjects: Image mosaicing -- Multimodal registration -- Hyperspectral imaging -- Gastroscopy; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2022.105234
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Gastric cancer is the second leading cause of cancer-related deaths worldwide. Early diagnosis significantly increases the chances of survival; therefore, improved assisted exploration and screening techniques are necessary. Previously, we made use of an augmented multi-spectral endoscope by inserting an optical probe into the instrumentation channel. However, the limited field of view and the lack of markings left by optical biopsies on the tissue complicate the navigation and revisit of the suspect areas probed in-vivo. In this contribution two innovative tools are introduced to significantly increase the traceability and monitoring of patients in clinical practice: (i) video mosaicing to build a more comprehensive and panoramic view of large gastric areas; (ii) optical biopsy targeting and registration with the endoscopic images. The proposed optical flow-based mosaicing technique selects images that minimize texture discontinuities and is robust despite the lack of texture and illumination variations. The optical biopsy targeting is based on automatic tracking of a free-marker probe in the endoscopic view using deep learning to dynamically estimate its pose during exploration. The accuracy of pose estimation is sufficient to ensure a precise overlapping of the standard white-light color image and the hyperspectral probe image, assuming that the small target area of the organ is almost flat. This allows the mapping of all spatio-temporally tracked biopsy sites onto the panoramic mosaic. Experimental validations are carried out from videos acquired on patients in hospital. The proposed technique is purely software-based and therefore easily integrable into clinical practice. It is also generic and compatible to any imaging modality that connects to a fiberscope. Graphical abstract: Image 1 Highlights: Novel optical flow mosaicing and optical biopsy site tracking methods are combined to generate enhanced endoscopic panoramas. Validation in clinical settings shows the technique capabilities to localize multispectral data during endoscopic screening. The resulting panoramic view presents a high visual coherence despite strong lighting variations and specular reflections. This technique is purely software-based, making it simple to integrate into endoscopic clinical practice.