Cross-domain assessment of deep learning-based alignment solutions for real-time 3D reconstruction. (October 2021)

• Record Type: Journal Article
• Authors: Lombardi, Marco; Savardi, Mattia; Signoroni, Alberto
• Is Part Of: Computers & graphics. Volume 99(2021)
• Page Start: 54
• Page End: 69
• Publication Date: 2021-10
• Subjects: Deep Learning -- Point-based models -- 3D Registration -- Real-time 3D Reconstruction -- Model Fine-tuning -- Benchmark Datasets; Computer graphics -- Periodicals; 006.6
• DOI: 10.1016/j.cag.2021.06.011
• Languages: English
• ISSNs: 0097-8493

Abstract

Highlights: First exploitation and demonstration of the role of 3D DL-based solutions to solve major issues affecting real-time 3D reconstruction systems. In-depth comparison of recent Deep-Learning based technologies for 3D view alignment. Extended benchmark on a new dataset (DenseMatch) representative of small-scale 3D acquisitions. Model generalizability tests and fine-tuning extensions of all the considered technologies. Graphical abstract: Abstract: Interesting deep learning solutions have been proposed recently to address different tasks along the 3D view alignment pipeline. However, a direct comparison among these technologies is still lacking, while their (possibly combined) potentials have yet to be extensively tested. This is especially true in cases where the focus is directed on diversified data and/or specific application requirements, such as the ones emerging in real-time 3D object reconstruction scenarios. This work is a first contribution in this direction since we perform an independent and extended comparison of the main deep learning-driven 3D view alignment solutions. We consider two relevant data types: data coming from commodity 3D sensors targeting indoor reconstruction applications, and denser data coming from a handheld 3D optical scanner, typically used for small-scale object reconstruction. While for the first scenario we refer to existing datasets, for the second setup we work on a new benchmarking dataset, namely DenseMatch. We run performance tests and extended comparisons, with different system configurations including model refinements, and we found solid evidence that the generalizability performance of deep learning systems for 3D alignment is critically linked to data features. Finally, we design and test the first integration of deep learning solutions into a baseline method for real-time 3D reconstruction, clearly demonstrating improved effectiveness in addressing and solving typical tracking and scan interruption issues arising in these demanding scenarios.