Two-view underwater 3D reconstruction for cameras with unknown poses under flat refractive interfaces. (September 2017)

• Record Type: Journal Article
• Authors: Kang, Lai; Wu, Lingda; Wei, Yingmei; Lao, Songyang; Yang, Yee-Hong
• Is Part Of: Pattern recognition. Volume 69(2017:Sep.)
• Page Start: 251
• Page End: 269
• Publication Date: 2017-09
• Subjects: Structure and Motion (SaM) -- Refractive distortion -- Underwater calibration -- Underwater 3D reconstruction -- Hybrid optimization; Pattern perception -- Periodicals; Perception des structures -- Périodiques; Patroonherkenning; 006.4
• DOI: 10.1016/j.patcog.2017.04.006
• Languages: English
• ISSNs: 0031-3203

Abstract

Highlights: The refractive distortion and epipolar errors are analyzed in detail using simulations. A new hybrid optimization framework for two-view underwater calibration with local sliding (LS) procedure based on bundle adjustment. A new fully automatic method for two-view underwater structure and motion estimation. The accuracy of 3D reconstruction is evaluated quantitatively by comparing the results to ground truth 3D models. The influence of non-zero thickness and non-parallel interface on 3D reconstruction is studied experimentally. Abstract: In an underwater imaging system, a refractive interface is introduced when a camera looks into the water-based environment, resulting in distorted images due to the refraction of light. Simply ignoring the refraction effect or using the lens radial distortion model causes erroneous 3D reconstruction. This paper deals with a general underwater imaging setup using two cameras, of which each camera is placed in a separate waterproof housing with a flat glass window. In order to handle refraction properly, a simplified refractive camera model is used in this paper. Based on two new concepts, namely the Ellipse of Refrax (EoR) and the Refractive Depth (RD) of a scene point, we derive two new formulations of the underwater known rotation structure and motion (SaM) problem. One gives a globally optimal solution and the other is robust to outliers. The constraint of known rotation is further relaxed by incorporating the robust known rotation SaM into a new hybrid optimization framework. Our method is able to simultaneously perform underwater camera calibration and 3D reconstruction automatically without using any calibration object or additional calibration device. In order to evaluate the performance and practical applicability of our method, extensive experiments using synthetic data, synthetically rendered images and real underwater images were carried out. The experimental results demonstrate that the proposed method can significantly improve the accuracy of the reconstructed 3D structure (within 0.78 mm for an object of dimension over 200 mm compared with the ground truth model captured by a land-based system) and of the system parameters for underwater applications. Compared with bundle adjustment using the refractive camera model initialized with traditional 3D reconstruction methods, our proposed optimization method has significantly better completeness and accuracy and lower 3D errors in the reconstructed models.