A Finite Horizon Markov Decision Process Based Reinforcement Learning Control of a Rapid Thermal Processing system. (August 2018) - Pradeep, D. John; Noel, Mathew Mithra
Highlights: A machine learning based solution is proposed for the challenging and economically important problem of controlling a rapid thermal processing system used for manufacturing large scale integrated circuits. The problem of controlling a rapid thermal processing system is formulated as a Finite Horizon Markov Decision Process (FHMDP). Simulation results indicate that the reinforcement learning based approach proposed in this paper significantly outperforms the state-of-the-art nonlinear control approach from literature. Three increasing complex strategies for controlling rapid thermal processing systems are proposed and explored. Abstract: Manufacture of ultra large-scale integrated circuits involves accurate control of a challenging nonlinear Rapid Thermal Processing (RTP) system. Precise control of temperature profile and rapid ramp-up and ramp-down rates demanded by a RTP system cannot be achieved with conventional control strategies due to nonlinear and multi time-scale effects. In this paper the control of a RTP system is reformulated as an optimal multi-step sequential decision problem using the framework of finite horizon Markov decision processes and solved using a Reinforcement Learning (RL) algorithm. Three increasingly complex RL based control strategies are explored and compared with the existing state-of-the-art approach for controlling RTPs. Simulation results indicate that the approach proposed in this paper achieves superior control of the temperature profile and ramp-up and ramp-down rates for the RTP system.