A Hardware-adaptive Deep Feature Matching Pipeline for Real-time 3D Reconstruction. (March 2021)

• Record Type: Journal Article
• Authors: Zheng, Shuai; Wang, Yabin; Li, Baotong; Li, Xin
• Is Part Of: Computer aided design. Volume 132(2021)
• Publication Date: 2021-03
• Subjects: Deep feature matching -- Neural architecture search -- Real-time 3D reconstruction -- Hardware-adaptive neural network optimization; Computer-aided design -- Periodicals; Engineering design -- Data processing -- Periodicals; Computer graphics -- Periodicals; Conception technique -- Informatique -- Périodiques; Infographie -- Périodiques; Computer graphics; Engineering design -- Data processing; Periodicals; Electronic journals; 620.00420285
• DOI: 10.1016/j.cad.2020.102984
• Languages: English
• ISSNs: 0010-4485

Abstract

Abstract: This paper presents a hardware-adaptive feature modeling framework to automatically generate and optimize deep neural networks to support real-time feature extraction and matching on a given hardware platform. This framework consists of a deep feature extraction and matching pipeline and a neural architecture search scheme, with which deep neural networks can be automatically generated and optimized according to given hardware to achieve reliable real-time feature matching. Built on our feature matching approach, we also developed a real-time 3D scene reconstruction pipeline that could run adaptively on hardware with different computational performance. We designed experiments to validate the proposed matching and reconstruction pipelines on hardware platforms with different performance. The results demonstrated our algorithm's effectiveness on both matching and reconstruction tasks. Highlights: A first automatic learning-based framework to generate neural network for real-time feature description. A neural network optimization system for platform-adaptive feature matching. A real-time 3D reconstruction system with state-of-the-art performance.