A cognitive assistance system with augmented reality for manual repair tasks with high variability based on the digital twin. (October 2022)

• Record Type: Journal Article
• Authors: Eversberg, Leon; Ebrahimi, Puya; Pape, Martin; Lambrecht, Jens
• Is Part Of: Manufacturing letters. Volume 34(2022)
• Page Start: 49
• Page End: 52
• Publication Date: 2022-10
• Subjects: Augmented reality -- Cognitive assistance system -- Digital twin -- Digital work instructions -- Maintenance repair overhaul; Manufacturing industries -- Periodicals; Production engineering -- Periodicals; Manufacturing industries; Periodicals; 670
• DOI: 10.1016/j.mfglet.2022.09.003
• Languages: English
• ISSNs: 2213-8463

Abstract

Highlights: We publish our ongoing research in the novel field of cognitive assistance systems for manual repair tasks in maintenance, repair and overhaul (MRO). Repair tasks in MRO depend on the highly variable condition of the repairable item. Thus, they require flexible work instructions in order to deal with the uncertainty in work scope. The proposed cognitive assistance system does not rely upon the manual authoring of standardized work instructions. Instead, it uses available object-specific information from the digital twin. We perform precise markerless pose estimation with two 3D cameras and use the resulting registration to display spatial information with multiple augmented reality display technologies. We showcase the system's potential on a typical MRO use case of a turbine blade repair process at a workstation for manual grinding. Abstract: Cognitive assistance systems with augmented reality are a promising solution to the increasing complexity in all industries. Our proposed assistance system is geared towards manual repair tasks with high variability in the maintenance, repair and overhaul industry. We use the digital twin for object-specific information and a human–machine interface that uses a web application for digital work instructions and augmented reality for spatial information. Context awareness of the assistance system is achieved by using two stationary 3D cameras and a barcode scanner. Camera images and point cloud data are used for precise markerless pose estimation and body tracking.