Unsupervised learning of depth estimation from imperfect rectified stereo laparoscopic images. (January 2022)

• Record Type: Journal Article
• Authors: Luo, Huoling; Wang, Congcong; Duan, Xingguang; Liu, Hao; Wang, Ping; Hu, Qingmao; Jia, Fucang
• Is Part Of: Computers in biology and medicine. Volume 140(2022)
• Publication Date: 2022-01
• Subjects: Unsupervised learning -- Depth estimation -- Stereo matching -- Laparoscopic image -- Imperfect rectified stereo images; Medicine -- Data processing -- Periodicals; Biology -- Data processing -- Periodicals; 610.285
• DOI: 10.1016/j.compbiomed.2021.105109
• Languages: English
• ISSNs: 0010-4825

Abstract

Abstract: Background: Learning-based methods have achieved remarkable performances on depth estimation. However, the premise of most self-learning and unsupervised learning methods is built on rigorous, geometrically-aligned stereo rectification. The performances of these methods degrade when the rectification is not accurate. Therefore, we explore an approach for unsupervised depth estimation from stereo images that can handle imperfect camera parameters. Methods: We propose an unsupervised deep convolutional network that takes rectified stereo image pairs as input and outputs corresponding dense disparity maps. First, a new vertical correction module is designed for predicting a correction map to compensate for the imperfect geometry alignment. Second, the left and right images, which are reconstructed based on the input image pair and corresponding disparities as well as the vertical correction maps, are regarded as the outputs of the generative term of the generative adversarial network (GAN). Then, the discriminator term of the GAN is used to distinguish the reconstructed images from the original inputs to force the generator to output increasingly realistic images. In addition, a residual mask is introduced to exclude pixels that conflict with the appearance of the original image in the loss calculation. Results: The proposed model is validated on the publicly available Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED) dataset and the average MAE is 3.054 mm. Conclusion: Our model can effectively handle imperfect rectified stereo images for depth estimation. Highlights: We propose an unsupervised depth estimation method for stereo endoscopic images. We introduce a new module to solve imperfect rectified stereo images matching. We develop a residual mask to exclude conflicted pixels during loss calculation. The proposed method achieves SOTA performance on publicly dataset.