A new approximate dynamic programming algorithm based on an actor–critic framework for optimal control of alkali–surfactant–polymer flooding. Issue 12 (2nd December 2019)

• Record Type: Journal Article
• Authors: Li, Shurong; Han, Lu; Ge, Yulei; Shi, Yuhuan
• Is Part Of: Engineering optimization. Volume 51:Issue 12(2019)
• Page Start: 2147
• Page End: 2168
• Publication Date: 2019-12-02
• Subjects: Approximate dynamic programming -- actor–critic framework -- ASP flooding -- action weighting -- eligibility trace; Engineering design -- Periodicals; Mathematical optimization -- Periodicals; 620.0042
• DOI: 10.1080/0305215X.2019.1570180
• Languages: English
• ISSNs: 0305-215X

Abstract

ABSTRACT: An approximate dynamic programming algorithm based on an actor–critic framework is proposed in this article. This algorithm takes the actor–critic framework as the basic framework, in which the actor and the critic are used to approximate the optimal value function and the control strategy, respectively. At first, the linear basis function approximator is used to approximate the value function. Then the method of basis function construction based on system characteristics is introduced. Furthermore, since the injection concentration of ASP flooding has a fixed interval, the action weighting method is adopted to restrict and approximate the optimal control action. The value function parameter and the two strategy parameters are updated by the gradient descent method. Meanwhile the eligibility trace is introduced to accelerate convergence. Finally, ASP flooding with four injection wells and nine production wells is used to test the effect of the proposed method.