Multi-objective reinforcement learning for fed-batch fermentation process control. (July 2022)

• Record Type: Journal Article
• Authors: Li, Dazi; Zhu, Fuqiang; Wang, Xiao; Jin, Qibing
• Is Part Of: Journal of process control. Volume 115(2022)
• Page Start: 89
• Page End: 99
• Publication Date: 2022-07
• Subjects: Multi-objective -- Reinforcement learning -- Fed-batch fermentation -- Process control; Process control -- Periodicals; Fabrication -- Contrôle -- Périodiques; Process control; Periodicals; Electronic journals; 660.281
• DOI: 10.1016/j.jprocont.2022.05.003
• Languages: English
• ISSNs: 0959-1524

Abstract

Abstract: Many real-world control problems involve conflicting objectives. For different objectives, it is necessary to obtain Pareto optimal solution sets for each one. Over recent years, multi-objective reinforcement learning (MORL) has been extensively studied to solve this problem. However, the multi-objective optimization problem of complex continuous control processes still requires exploration. Soft proximal policy optimization algorithms have also been proposed to be combined with a hybrid weight-generation method for application to find the Pareto front approximation of the fed-batch fermentation process. This algorithm intends to initially find a single policy for the multi-objective reinforcement learning problem. A hybrid weight-generation method is then used to change the weights between different objectives to find a set of Pareto optimal solutions. In addition, we analyzed the mechanism of the fed-batch process, established the kinetic model, and designed the experimental environment based on the OpenAI Gym library. Experimental results showed that the proposed algorithm is effective and efficient in approaching the Pareto front of the fed-batch fermentation problem. Highlights: A multi-objective optimization problem of fed-batch fermentation process is explored. Soft PPO algorithm combined with a hybrid weight-generation method is proposed. Experiments show the efficiency in approaching the Pareto front of the fermentation problem.