A fast and flexible HW/SW co-processing framework for Time-of-Flight 3D imaging. (February 2018) - Druml, Norbert; Ehrenhoefer, Christoph; Bell, Walter; Gailer, Christian; Plank, Hannes; Herndl, Thomas; Holweg, Gerald
Abstract: Time-of-Flight 3D imaging, using the indirect measuring method that employs photonic mixing devices, increases in popularity. This is due to the recent availability of accurate and miniaturized Time-of-Flight cameras that can be integrated into small embedded devices. However, providing a system comprising a camera and hardware-accelerated processing, which is useable for various application types with diametrically opposed use-case requirements, is not trivial. Here we introduce a fast and flexible hardware/software co-processing framework for Time-of-Flight 3D imaging, which can be used for various kinds of use-case applications. This is achieved in particular by providing fine-grained hardware-acceleration for Time-of-Flight algorithms and computer vision applications. Furthermore, this hardware/software framework enables and features state-of-the-art multi-core, mixed-criticality, and real-time concepts, which is essential for critical applications in the automotive and industrial domains. We demonstrate the feasible implementation and integration of the framework by means of a mixed-critical automotive system demonstrator. This demonstrator enables various types of use-case scenarios, such as gesture recognition or indoor navigation, by employing the AURIX automotive processor from Infineon Technologies and the Xilinx Zynq platform. For instance, if a typical gesture recognition use-case is addressed, the framework is capable to provide high-quality depth images and 3D-point cloud data with nearly 100 FPS and introduces an average calculation error of only 0.08 mm. Most importantly, a high level of flexibility in software and hardware is preserved in order to support other use-case scenarios at the same time. Highlights: It introduces a novel flexible hardware/software framework for Time-of-Flight 3D image processing. It shows the feasible realization of the flexible Time-of-Flight processing concept by means of a demonstrator targeting mixed-critical applications. It provides detailed evaluation results concerning various performance parameters and utilization of resources.