A fast and flexible HW/SW co-processing framework for Time-of-Flight 3D imaging. (February 2018)

• Record Type: Journal Article
• Authors: Druml, Norbert; Ehrenhoefer, Christoph; Bell, Walter; Gailer, Christian; Plank, Hannes; Herndl, Thomas; Holweg, Gerald
• Is Part Of: Microelectronics and reliability. Volume 81(2018)
• Page Start: 64
• Page End: 76
• Publication Date: 2018-02
• Subjects: Time-of-Flight 3D imaging -- Hardware/software co-design -- FPGA; Electronic apparatus and appliances -- Reliability -- Periodicals; Miniature electronic equipment -- Periodicals; Appareils électroniques -- Fiabilité -- Périodiques; Équipement électronique miniaturisé -- Périodiques; Electronic apparatus and appliances -- Reliability; Miniature electronic equipment; Periodicals; 621.3815
• DOI: 10.1016/j.microrel.2017.12.004
• Languages: English
• ISSNs: 0026-2714

Abstract

Abstract: Time-of-Flight 3D imaging, using the indirect measuring method that employs photonic mixing devices, increases in popularity. This is due to the recent availability of accurate and miniaturized Time-of-Flight cameras that can be integrated into small embedded devices. However, providing a system comprising a camera and hardware-accelerated processing, which is useable for various application types with diametrically opposed use-case requirements, is not trivial. Here we introduce a fast and flexible hardware/software co-processing framework for Time-of-Flight 3D imaging, which can be used for various kinds of use-case applications. This is achieved in particular by providing fine-grained hardware-acceleration for Time-of-Flight algorithms and computer vision applications. Furthermore, this hardware/software framework enables and features state-of-the-art multi-core, mixed-criticality, and real-time concepts, which is essential for critical applications in the automotive and industrial domains. We demonstrate the feasible implementation and integration of the framework by means of a mixed-critical automotive system demonstrator. This demonstrator enables various types of use-case scenarios, such as gesture recognition or indoor navigation, by employing the AURIX automotive processor from Infineon Technologies and the Xilinx Zynq platform. For instance, if a typical gesture recognition use-case is addressed, the framework is capable to provide high-quality depth images and 3D-point cloud data with nearly 100 FPS and introduces an average calculation error of only 0.08 mm. Most importantly, a high level of flexibility in software and hardware is preserved in order to support other use-case scenarios at the same time. Highlights: It introduces a novel flexible hardware/software framework for Time-of-Flight 3D image processing. It shows the feasible realization of the flexible Time-of-Flight processing concept by means of a demonstrator targeting mixed-critical applications. It provides detailed evaluation results concerning various performance parameters and utilization of resources.