Mutual information-enhanced digital twin promotes vision-guided robotic grasping. (April 2022) - Hu, Fuwen
Graphical abstract: Highlights: An embodied decision-making method is presented by the marriage between digital twin and information-theoretic approach. The mutual information-enhanced digital twin of conveyor-based visual detection is proposed. The adaptability of the vision-guided robotic grasping system to the real-world randomness has been clearly increased. Abstract: Vision-guided learning for autonomous robotic manipulations is a wide-ranging and high-impact topic in the context of smart manufacturing. Most learning strategies are object-centered or prior information-dependent, which likely lead to the problems of generalization across objects or scenes. To alleviate this, this work presented an embodiment decision-making method by the marriage between the digital twin epistemology and information-theoretic approach. The initial insight was that the mutual information generated in the interactions between the available vision models and real-world perceptions could decrease the uncertainty of sensing-action processes. Further, the real-time interactive information gains and visual templates constitute the digital twin through bidirectional data flowing and real-time optimization. As a demonstration of concept, on the conveyor-based and vision-guided robotic grasping platform, the robotic grasping experiments of freely placed and moving parts were performed. Experimental results indicated that the autonomous and real-time optimization of the conveyor-based and vision-guided robotic grasping system happens and the adaptability to the real-world changes had been clearly increased. This research suggested that the representation and dynamic capture of the complex interactions between both sides of cyber-physical system could generate new possibilities to the evolution of decision-making paradigm in more complex industrial processes.