Imitation of piping warm-up operation and estimation of operational intention by inverse reinforcement learning. (February 2023)

• Record Type: Journal Article
• Authors: Nakagawa, Yosuke; Ono, Hitoi; Hazui, Yusuke; Arai, Sachiyo
• Is Part Of: Journal of process control. Volume 122(2023)
• Page Start: 41
• Page End: 48
• Publication Date: 2023-02
• Subjects: Inverse reinforcement learning -- Plant operation -- Estimation of operational intention; Process control -- Periodicals; Fabrication -- Contrôle -- Périodiques; Process control; Periodicals; Electronic journals; 660.281
• DOI: 10.1016/j.jprocont.2022.12.010
• Languages: English
• ISSNs: 0959-1524

Abstract

Abstract: In this study, AIRL, which is a method of adversarial inverse reinforcement learning, was applied to the warm-up operation of steam pipes. We tried to imitate the operation of the expert and guessed the operation intention. As a result, we were able to learn the stepwise expert operation of heating → pressurizing → reheating performed by trials equivalent to 2800 times in the plant operation. In addition, by simultaneous learning that alternately gives two expert data of summer and winter, we got a robust policy function applicable to operating conditions of different seasons. Furthermore, by visualizing the state value function and reward function of the discriminator, it was found that the policy learned under a single condition in summer cannot be applied in winter because it has not been able to learn the operation to maintain the degree of superheat of steam immediately after the start of heating. In the field of process control, it was confirmed that AIRL, which divides the value function and the reward function and can transfer and visualize the reward function, is effective for imitating expert operations and guessing their intentions.