Event-based tracking of human hands. Issue 4 (22nd September 2021)

• Record Type: Journal Article
• Authors: Duarte, Laura; Safeea, Mohammad; Neto, Pedro
• Is Part Of: Sensor review. Volume 41:Issue 4(2021)
• Page Start: 382
• Page End: 389
• Publication Date: 2021-09-22
• Subjects: Dynamic vision sensor -- Event camera -- Human hands tracking; Sensor systems -- Periodicals; Detectors -- Industrial applications -- Periodicals; Engineering instruments -- Periodicals; 681.2
• DOI: 10.1108/SR-03-2021-0095
• Languages: English
• ISSNs: 0260-2288

Abstract

Abstract : Purpose: This paper proposes a novel method for human hands tracking using data from an event camera. The event camera detects changes in brightness, measuring motion, with low latency, no motion blur, low power consumption and high dynamic range. Captured frames are analysed using lightweight algorithms reporting three-dimensional (3D) hand position data. The chosen pick-and-place scenario serves as an example input for collaborative human–robot interactions and in obstacle avoidance for human–robot safety applications. Design/methodology/approach: Events data are pre-processed into intensity frames. The regions of interest (ROI) are defined through object edge event activity, reducing noise. ROI features are extracted for use in-depth perception. Findings: Event-based tracking of human hand demonstrated feasible, in real time and at a low computational cost. The proposed ROI-finding method reduces noise from intensity images, achieving up to 89% of data reduction in relation to the original, while preserving the features. The depth estimation error in relation to ground truth (measured with wearables), measured using dynamic time warping and using a single event camera, is from 15 to 30 millimetres, depending on the plane it is measured. Originality/value: Tracking of human hands in 3 D space using a single event camera data and lightweight algorithms to define ROI features (hands tracking in space).