A Finite Horizon Markov Decision Process Based Reinforcement Learning Control of a Rapid Thermal Processing system. (August 2018)

• Record Type: Journal Article
• Authors: Pradeep, D. John; Noel, Mathew Mithra
• Is Part Of: Journal of process control. Volume 68(2018)
• Page Start: 218
• Page End: 225
• Publication Date: 2018-08
• Subjects: Reinforcement Learning -- Rapid Thermal Processing -- Nonlinear control -- Markov Decision Process -- Process control -- Multivariable control; Process control -- Periodicals; Fabrication -- Contrôle -- Périodiques; Process control; Periodicals; Electronic journals; 660.281
• DOI: 10.1016/j.jprocont.2018.06.002
• Languages: English
• ISSNs: 0959-1524

Abstract

Highlights: A machine learning based solution is proposed for the challenging and economically important problem of controlling a rapid thermal processing system used for manufacturing large scale integrated circuits. The problem of controlling a rapid thermal processing system is formulated as a Finite Horizon Markov Decision Process (FHMDP). Simulation results indicate that the reinforcement learning based approach proposed in this paper significantly outperforms the state-of-the-art nonlinear control approach from literature. Three increasing complex strategies for controlling rapid thermal processing systems are proposed and explored. Abstract: Manufacture of ultra large-scale integrated circuits involves accurate control of a challenging nonlinear Rapid Thermal Processing (RTP) system. Precise control of temperature profile and rapid ramp-up and ramp-down rates demanded by a RTP system cannot be achieved with conventional control strategies due to nonlinear and multi time-scale effects. In this paper the control of a RTP system is reformulated as an optimal multi-step sequential decision problem using the framework of finite horizon Markov decision processes and solved using a Reinforcement Learning (RL) algorithm. Three increasingly complex RL based control strategies are explored and compared with the existing state-of-the-art approach for controlling RTPs. Simulation results indicate that the approach proposed in this paper achieves superior control of the temperature profile and ramp-up and ramp-down rates for the RTP system.