A fast decision-making method for process planning with dynamic machining resources via deep reinforcement learning. (January 2021)

• Record Type: Journal Article
• Authors: Wu, Wenbo; Huang, Zhengdong; Zeng, Jiani; Fan, Kuan
• Is Part Of: Journal of manufacturing systems. Volume 58(2021)Part A
• Page Start: 392
• Page End: 411
• Publication Date: 2021-01
• Subjects: Deep reinforcement learning -- Computer-aided process planning -- Combinatorial optimization -- Decision making; Manufacturing processes -- Periodicals; Production engineering -- Data processing -- Periodicals; Robots, Industrial -- Periodicals; Production, Technique de la -- Informatique -- Périodiques; Robots industriels -- Périodiques; Electronic journals; 670.42
• DOI: 10.1016/j.jmsy.2020.12.015
• Languages: English
• ISSNs: 0278-6125

Abstract

Highlights: This paper solves a complex and dynamic decision-making problem with multiple constraints by using deep reinforcement learning (DRL) method. The DRL method significantly reduces the response time, such that it can cope with the dynamic change of the resource availability. For implementing the DRL method, a Markov decision-making procedure is carried out by integrating two different types of decisions into one. The process planning policy is oriented to long-term goals based on reward function. Abstract: Mass customized production brings great uncertainty to the computer-aided process planning (CAPP). Current CAPP methods based on heuristic optimization assume in advance that manufacturing resources are static and make a deterministic plan that cannot cope with the uncertainty of the manufacture environment. As a promising method in solving complex and dynamic decision-making problems, deep reinforcement learning is employed in this paper for process planning, aiming at promoting the response speed by exploiting the reusability and expandability of past decision-making experiences. To simplify the decision procedure, two different types of decisions, operation sequencing and resource selection, are fused into one by integrating environment states and agent behaviors in a matrix manner. Then, a masking algorithm is developed to screen out currently inexecutable machining operations at each decision step and process planning datasets are generated for training and testing according to the actual processing logic. Next, the Monte Carlo method and the deep learning algorithm are utilized to evaluate and improve the process policy, respectively. Finally, the searching capability of the proposed method for both static and dynamic manufacturing resources are tested in case studies, and the results are discussed. It is shown that the proposed approach can solve the planning problem more efficiently compared with current optimization-based approaches.