Assessment of the handcart pushing and pulling safety by using deep learning 3D pose estimation and IoT force sensors. (30th November 2021)

• Record Type: Journal Article
• Authors: Vukicevic, Arso M.; Macuzic, Ivan; Mijailovic, Nikola; Peulic, Aleksandar; Radovic, Milos
• Is Part Of: Expert systems with applications. Volume 183(2021)
• Publication Date: 2021-11-30
• Subjects: Deep learning -- Ergonomics -- Pushing and pulling -- Handcart -- Unsafe acts; Expert systems (Computer science) -- Periodicals; Systèmes experts (Informatique) -- Périodiques; Electronic journals; 006.33
• DOI: 10.1016/j.eswa.2021.115371
• Languages: English
• ISSNs: 0957-4174

Abstract

Graphical abstract: Highlights: Manual safety management of pushing and pulling (P&P) tasks is inefficient. IoT force sensors were used to assess P&P forces. Safety of P&P acts was assessed from 3D poses obtained with the VIBE algorithm. Besides increased forces, unsafe P&P acts are correlated with the P&P momentum. Future studies should account for turn-points and loading/unloading of cargo. Abstract: Pushing and pulling (P&P) are common and repetitive tasks in industry, which non-ergonomic execution is among major causes of musculoskeletal disorders (MSD). The current safety management of P&P assumes restrictions of maximal weight, distance, height – while variable individual parameters (such as the P&P pose ergonomic) remain difficult to account for with the standardized guides. Since manual detection of unsafe P&P acts is subjective and inefficient, the aim of this study was to utilize IoT force sensors and IP cameras to detect unsafe P&P acts timely and objectively. Briefly, after the IoT module detects moments with increased P&P forces, the assessment of pose ergonomics was performed from the employee pose reconstructed with the VIBE algorithm. The experiments showed that turn-points correspond to the high torsion of torso, and that in such moments poses are commonly non ergonomic (although P&P forces are below values defined as critical in previous studies – their momentum cause serious load on the human body). Moreover, the analysis revealed that the loading/unloading of a cargo are also moments of frequent unsafe P&P acts – although they are commonly neglected when studying P&P. The experimental validation of the solution showed good agreement with motion sensors and high potential for monitoring and improving P&P workplace safety. Accordingly, future research will be directed towards: 1) acquisition of P&P data sets for direct recognition and classification of unsafe P&P acts; 2) incorporation of wearable sensors (EMG and EEG) for detecting fatigue and decrease of physical abilities.