Real-time monitoring for manual operations with machine vision in smart manufacturing. (October 2022) - Lou, Ping; Li, Ji; Zeng, YuHang; Chen, Bing; Zhang, Xiaomei
Abstract: Online real-time production process monitoring is the basis for intelligent manufacturing refinement management. This paper proposes a contactless monitoring framework with machine vision to realize real-time production/assembly process monitoring and repetition counting. In addition, a two-stage approach is presented to solve the issue of manual operation recognition and counting by pieces. In the first stage, a high-efficient method for object detection and classification, YOLOv4-tiny, is used to recognize manual operations/sub-operations and classify the sub-operations of the operations using monitoring video. And a Sliding Window Counter algorithm is utilized to find appropriate boundary points of different categories of the sub-operations for counting in the second stage. A dataset of repetitive manual operations is constructed for training this approach. The experiment results on the production/assembly line operations indicate that the approach can identify and count worker operations in real time. Highlights: The contactless and online perception method is presented to monitor manual operations. The lightweight object detection method is used to recognize the sub-operations at high-speed. A state detecting method with a sliding window is put forward for counting by pieces in real-time.