NerveStitcher: Corneal confocal microscope images stitching with neural networks. (December 2022)

• Record Type: Journal Article
• Authors: Li, Guangxu; Li, Tianyu; Li, Fangting; Zhang, Chen
• Is Part Of: Computers in biology and medicine. Volume 151:Part B(2022)
• Publication Date: 2022-12
• Subjects: Images stitching -- In vivo confocal microscopy -- Neural networks -- Corneal subbasal nerve plexus; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2022.106303
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Corneal nerves are of great interest to clinicians and scientists due to their potential for the diagnosis of early neurological disorders. In vivo confocal microscopy (IVCM) has been used as a novel and reliable tool for observing and quantifying corneal sub-basal nerves. Creating a wide-field montage of the nerve plexus from a large amount of IVCM images facilitates the measurement of corneal nerve morphology. In this paper, we propose a fully automatic image stitching method using neural networks. Firstly, we extend a self-supervised point detector to find the feature points on IVCM images. Then a flexible points correspondence based on the attention mechanism is developed for partial assignment of image pair. The scattered IVCM images are consequently integrated and fused according to the local offsets. We experimented with our method on 30 sets of IVCM images. Compared to conventional methods, our method improves matching accuracy and significantly reduces processing time. And by calculating the morphological parameters of the corneal nerve for both single images and stitched images, our method can evaluate the corneal nerve of patients more accurately and reliably. The implemented code is available at https://github.com/LiTianYu6/NerveStitcher . Highlights: The first deep learning based IVCM image stitching method is designed, called NerveStitcher. It consists of a convolutional neural network and a graph convolutional neural network with good efficiency and accuracy. The feature point detection uses self-supervised learning without human labeling. Self-attention and cross-attention mechanisms are introduced for feature point correspondence. A novel vignetting correction is proposed in image pre-processing to prevent unsmooth transitions between image stitching seams.