Optimization of oxygen system scheduling in hybrid action space based on deep reinforcement learning. (March 2023)

• Record Type: Journal Article
• Authors: Li, Lijuan; Yang, Xue; Yang, Shipin; Xu, Xiaowei
• Is Part Of: Computers & chemical engineering. Volume 171(2023)
• Publication Date: 2023-03
• Subjects: Oxygen system for steel enterprises -- Hybrid action space -- Deep Q network -- Actor-critic algorithm; Chemical engineering -- Data processing -- Periodicals; 660.0285
• DOI: 10.1016/j.compchemeng.2023.108168
• Languages: English
• ISSNs: 0098-1354

Abstract

Highlights: A hybrid actor-critic algorithm is proposed to solve the optimization problem of the oxygen system. The discrete and continuous action network is designed to address scheduling problems with hybrid actors. An additional reward function and the correlation matrix are constructed to improve the accuracy. Experiments with multiple groups employing the real data from steel enterprises are carried out. Abstract: The simultaneous consideration of discrete and continuous variables including equipment start-up, shutdown, oxygen output and dissipation, is required in the optimal problem of oxygen system scheduling in steel enterprises. To solve this problem involving hybrid variables, a hybrid actor-critic (HAC) algorithm is proposed in this paper. The proposed algorithm subdivides the action space into a discrete and continuous action space and evaluates the policy through the improved Q function. Besides, the correlation matrix is constructed to address the correspondence between hybrid actions. As a result, the generation of spurious gradients that lead to suboptimal action selection is prevented. To verify the effectiveness of the proposed approach, experiments with multiple groups employing the real data from steel enterprises are carried out. The results demonstrate that such a practice-based solution successfully resolves the oxygen scheduling problem and simultaneously improves the reward and algorithm convergence speed.