A fuzzy hierarchical reinforcement learning based scheduling method for semiconductor wafer manufacturing systems. (October 2021)

• Record Type: Journal Article
• Authors: Wang, Junliang; Gao, Pengjie; Zheng, Peng; Zhang, Jie; Ip, W.H.
• Is Part Of: Journal of manufacturing systems. Volume 61(2021)
• Page Start: 239
• Page End: 248
• Publication Date: 2021-10
• Subjects: Scheduling -- Reinforcement learning -- Fuzzy -- Manufacturing system; Manufacturing processes -- Periodicals; Production engineering -- Data processing -- Periodicals; Robots, Industrial -- Periodicals; Production, Technique de la -- Informatique -- Périodiques; Robots industriels -- Périodiques; Electronic journals; 670.42
• DOI: 10.1016/j.jmsy.2021.08.008
• Languages: English
• ISSNs: 0278-6125

Abstract

Highlights: The scheduling method for the whole semiconductor wafer manufacturing systems. A hierarchical reinforcement learning model is presented to optimize the scheduling problems. An actor-critic agent is improved to optimize the control instruction. Fuzzy learning is designed to reduce the uncertainty of expected finishing time. Abstract: Scheduling semiconductor wafer manufacturing systems has been viewed as one of the most challenging optimization problems owing to the complicated constraints, and dynamic system environment. This paper proposes a fuzzy hierarchical reinforcement learning (FHRL) approach to schedule a SWFS, which controls the cycle time (CT) of each wafer lot to improve on-time delivery by adjusting the priority of each wafer lot. To cope with the layer correlation and wafer correlation of CT due to the re-entrant process constraint, a hierarchical model is presented with a recurrent reinforcement learning (RL) unit in each layer to control the corresponding sub-CT of each integrated circuit layer. In each RL unit, a fuzzy reward calculator is designed to reduce the impact of uncertainty of expected finishing time caused by the rematching of a lot to a delivery batch. The results demonstrate that the mean deviation (MD) between the actual and expected completion time of wafer lots under the scheduling of the FHRL approach is only about 30 % of the compared methods in the whole SWFS.