Reinforcement learning for soft sensor design through autonomous cross-domain data selection. (May 2023)

• Record Type: Journal Article
• Authors: Xie, Junyao; Dogru, Oguzhan; Huang, Biao; Godwaldt, Chris; Willms, Brett
• Is Part Of: Computers & chemical engineering. Volume 173(2023)
• Publication Date: 2023-05
• Subjects: Soft sensor -- Reinforcement learning -- Data selection -- Domain adaptation; Chemical engineering -- Data processing -- Periodicals; 660.0285
• DOI: 10.1016/j.compchemeng.2023.108209
• Languages: English
• ISSNs: 0098-1354

Abstract

Abstract: Data-driven soft sensors have been extensively applied in the process industry for quality variable estimation. It is challenging to build reliable soft sensors for complex industrial processes under new operating conditions where available data are limited. To overcome this issue, we leverage samples from source domains, formulate the sample selection and soft sensor problem of the target domain as a Markov decision process, and solve this cross-domain soft sensor problem by proposing a reinforcement learning framework. Specifically, we propose an asynchronous advantage selector-actor-critic method for cross-domain sample selection and soft sensor design. The transferability of source-domain samples to the target domain is determined by the proposed method. The correlation and estimation error metrics are incorporated into the reward function for the performance-driven design. An extension to feature data selection is also proposed. The applicability of the proposed methods is demonstrated via a simulation study and an industrial case study. Highlights: A reinforcement learning frame for autonomous cross-domain soft sensing is proposed. A Markov decision process formulation is developed. Asynchronous advantage selector-actor-critic methods are proposed. A novel reward is defined by combining the correlation and prediction error metrics. A numerical simulation and an industrial case study verify the proposed methods.