A digital twin to train deep reinforcement learning agent for smart manufacturing plants: Environment, interfaces and intelligence. (January 2021)

• Record Type: Journal Article
• Authors: Xia, Kaishu; Sacco, Christopher; Kirkpatrick, Max; Saidy, Clint; Nguyen, Lam; Kircaliali, Anil; Harik, Ramy
• Is Part Of: Journal of manufacturing systems. Volume 58(2021)Part B
• Page Start: 210
• Page End: 230
• Publication Date: 2021-01
• Subjects: Smart manufacturing systems -- Robotics -- Artificial intelligence -- Digital transformation -- Virtual commissioning; Manufacturing processes -- Periodicals; Production engineering -- Data processing -- Periodicals; Robots, Industrial -- Periodicals; Production, Technique de la -- Informatique -- Périodiques; Robots industriels -- Périodiques; Electronic journals; 670.42
• DOI: 10.1016/j.jmsy.2020.06.012
• Languages: English
• ISSNs: 0278-6125

Abstract

Highlights: Methods to construct high-fidelity digital twin for automation systems is introduced. Network of interfaces enabling communications among system components is built. Manufacturing intelligence is realized by training Deep Reinforcement Learning. A smart dynamic scheduler is developed for continuous process optimization. Abstract: Filling the gaps between virtual and physical systems will open new doors in Smart Manufacturing. This work proposes a data-driven approach to utilize digital transformation methods to automate smart manufacturing systems. This is fundamentally enabled by using a digital twin to represent manufacturing cells, simulate system behaviors, predict process faults, and adaptively control manipulated variables. First, the manufacturing cell is accommodated to environments such as computer-aided applications, industrial Product Lifecycle Management solutions, and control platforms for automation systems. Second, a network of interfaces between the environments is designed and implemented to enable communication between the digital world and physical manufacturing plant, so that near-synchronous controls can be achieved. Third, capabilities of some members in the family of Deep Reinforcement Learning (DRL) are discussed with manufacturing features within the context of Smart Manufacturing. Trained results for Deep Q Learning algorithms are finally presented in this work as a case study to incorporate DRL-based artificial intelligence to the industrial control process. As a result, developed control methodology, named Digital Engine, is expected to acquire process knowledges, schedule manufacturing tasks, identify optimal actions, and demonstrate control robustness. The authors show that integrating a smart agent into the industrial platforms further expands the usage of the system-level digital twin, where intelligent control algorithms are trained and verified upfront before deployed to the physical world for implementation. Moreover, DRL approach to automated manufacturing control problems under facile optimization environments will be a novel combination between data science and manufacturing industries.