Mutual information-enhanced digital twin promotes vision-guided robotic grasping. (April 2022)

• Record Type: Journal Article
• Authors: Hu, Fuwen
• Is Part Of: Advanced engineering informatics. Volume 52(2022)
• Publication Date: 2022-04
• Subjects: Robotic grasping -- Machine vision -- Digital twin -- Mutual information -- Smart manufacturing; Computer-aided engineering -- Periodicals; Engineering -- Data processing -- Periodicals; 620.00285
• DOI: 10.1016/j.aei.2022.101562
• Languages: English
• ISSNs: 1474-0346

Abstract

Graphical abstract: Highlights: An embodied decision-making method is presented by the marriage between digital twin and information-theoretic approach. The mutual information-enhanced digital twin of conveyor-based visual detection is proposed. The adaptability of the vision-guided robotic grasping system to the real-world randomness has been clearly increased. Abstract: Vision-guided learning for autonomous robotic manipulations is a wide-ranging and high-impact topic in the context of smart manufacturing. Most learning strategies are object-centered or prior information-dependent, which likely lead to the problems of generalization across objects or scenes. To alleviate this, this work presented an embodiment decision-making method by the marriage between the digital twin epistemology and information-theoretic approach. The initial insight was that the mutual information generated in the interactions between the available vision models and real-world perceptions could decrease the uncertainty of sensing-action processes. Further, the real-time interactive information gains and visual templates constitute the digital twin through bidirectional data flowing and real-time optimization. As a demonstration of concept, on the conveyor-based and vision-guided robotic grasping platform, the robotic grasping experiments of freely placed and moving parts were performed. Experimental results indicated that the autonomous and real-time optimization of the conveyor-based and vision-guided robotic grasping system happens and the adaptability to the real-world changes had been clearly increased. This research suggested that the representation and dynamic capture of the complex interactions between both sides of cyber-physical system could generate new possibilities to the evolution of decision-making paradigm in more complex industrial processes.