Control of a bioreactor using a new partially supervised reinforcement learning algorithm. (September 2018)

• Record Type: Journal Article
• Authors: Pandian, B. Jaganatha; Noel, Mathew Mithra
• Is Part Of: Journal of process control. Volume 69(2018)
• Page Start: 16
• Page End: 29
• Publication Date: 2018-09
• Subjects: Machine learning -- Reinforcement learning -- Neural networks -- Nonlinear control -- Bioreactor control -- Interacting multiple tank control; Process control -- Periodicals; Fabrication -- Contrôle -- Périodiques; Process control; Periodicals; Electronic journals; 660.281
• DOI: 10.1016/j.jprocont.2018.07.013
• Languages: English
• ISSNs: 0959-1524

Abstract

Highlights: A new nonlinear control strategy that synergistically combines supervised and reinforcement learning based control paradigms is proposed. A machine learning based control strategy is proposed for the challenging and economically important bioreactor control problem. Simulation results indicate that the proposed partially supervised RL control strategy is superior to both pure RL and inverse model ANN nonlinear control strategies for bioreactor control on a wide variety of performance metrics. Experiment results demonstrating the control of a quadruple interacting tank system using the proposed PSRL algorithm are presented. Experimental as well as simulation results clearly demonstrate the superior convergence of the proposed PSRL algorithm. Abstract: In recent years, researchers have explored the application of Reinforcement Learning (RL) and Artificial Neural Networks (ANNs) to the control of complex nonlinear and time varying industrial processes. However RL algorithms use exploratory actions to learn an optimal control policy and converge slowly while popular inverse model ANN based control strategies require extensive training data to learn the inverse model of complex nonlinear systems. In this paper a novel approach that avoids the need for extensive training data to construct an exact inverse model in the inverse ANN approach, the need for an exact and stable inverse to exist and the need for exhaustive and costly exploration in pure RL based strategies is proposed. In this approach an initial approximate control policy learnt by an artificial neural network is refined using a reinforcement learning strategy. This Partially Supervised Reinforcement Learning (PSRL) strategy is applied to the economically important problem of control of a semi-continuous batch-fed bioreactor used for yeast fermentation. The bioreactor control problem is formulated as a Markov Decision Process (MDP) and solved using pure RL and PSRL algorithms. Model based and model-free RL control experiments and simulations are used to demonstrate the superior performance of the PSRL strategy compared to the pure RL and inverse model ANN based control strategies on a variety of performance metrics.