Markerless cooperative augmented reality-based smart manufacturing double-check system: Case of safe PCBA inspection following automatic optical inspection. (August 2020)

• Record Type: Journal Article
• Authors: Runji, Joel Murithi; Lin, Chyi-Yeu
• Is Part Of: Robotics and computer-integrated manufacturing. Volume 64(2020)
• Publication Date: 2020-08
• Subjects: Augmented reality -- Inspection -- Cooperative -- Printed circuit board assembly -- Smart manufacturing -- Optical see-through; Robots, Industrial -- Periodicals; Computer integrated manufacturing systems -- Periodicals; Robotics -- Periodicals; Robots industriels -- Périodiques; Productique -- Périodiques; Robotique -- Périodiques; 670.285
• DOI: 10.1016/j.rcim.2020.101957
• Languages: English
• ISSNs: 0736-5845

Abstract

Highlights: The developed system's registration is robust to board size. Head-mount visualization is appropriate for inspection compared to handheld based. Multi-stage rendering of images is suitable in the vergence-accomodation conflict region. Superposing position with orientation information provides more efficient guidance. Contextual displays are more appropriate than static ones during inspection. Abstract: Augmented reality (AR) is a key technology anchored towards realizing Industry 4.0 smart manufacturing aims. In manufacturing inspection, AR has previously been employed to support operators in assessing thickness of manufactured parts, ship building, aircraft subassemblies and printed circuit board assemblies (PCBAs) with or without external markers in standalone head-mounted (HMD) or handheld systems. These AR systems often use optical see-through (OST) or video see-through (VST) technologies. Currently, cyber physical integration of processes in an industrial system synergizes production through increased efficiency and improved quality while facilitating customization. An operator manually double-checking/inspecting a product that has previously been automatically inspected can better rely on this existing defect location information once it is contextually and spatially overlaid within their field of view to intuitively and efficiently execute the task. Additional interactive information provided can quickly reorient a user, provide necessary contextual reference, and monitor progress of the task while alerting them of their safety. In this study, adopting an automatic optical inspection (AOI) wirelessly aided HMD-OST AR-based manual inspection system requiring no external markers for contextual registration is a promising direction towards smart and safe PCBA manufacturing in line with Industry 4.0. Animated rectangular bracket of high color contrast is employed to localize the spawned AOI defect point, an arrow overlaid with distance text provides user guidance, a contextual resizable image facilitates user double-checking and a progress bar tracks the inspection progress while monitoring tracking state. We evaluate the developed system's robustness, effectiveness of display technology used, suitability of contextual against static display modes, and display technology influence on various inspection attributes in a user study. Results demonstrate the system to be robust, OST-HMD outperforms handheld VST devices, contextual display mode to be significantly preferred to static mode, and display technology employed has no significant influence on the inspection attributes. Finally, registration precision results demonstrate usability of the system while superposition of distance to orientation information raises the inspection rate of PCBAs.