An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system. (November 2019)

• Record Type: Journal Article
• Authors: Pang, Ji-Wei; Yang, Shan-Shan; He, Lei; Chen, Yi-Di; Cao, Guang-Li; Zhao, Lei; Wang, Xin-Yu; Ren, Nan-Qi
• Is Part Of: Chemosphere. Volume 234(2019)
• Page Start: 893
• Page End: 901
• Publication Date: 2019-11
• Subjects: Biological phosphorus removal -- Machine learning -- Improved QL algorithm -- Real-time control strategy -- ASM2d -- Fluctuant influent loads; Pollution -- Periodicals; Pollution -- Physiological effect -- Periodicals; Environmental sciences -- Periodicals; Atmospheric chemistry -- Periodicals; 551.511
• DOI: 10.1016/j.chemosphere.2019.06.103
• Languages: English
• ISSNs: 0045-6535

Abstract

Abstract: Biological phosphorus removal (BPR) is an economical and sustainable processes for the removal of phosphorus (P) from wastewater, achieved by recirculating activated sludge through anaerobic and aerobic (An/Ae) processes. However, few studies have systematically analyzed the optimal hydraulic retention times (HRTs) in anaerobic and aerobic reactions, or whether these are the most appropriate control strategies. In this study, a novel optimization methodology using an improved Q-learning (QL) algorithm was developed, to optimize An/Ae HRTs in a BPR system. A framework for QL-based BPR control strategies was established and the improved Q function, Q t + 1 ( s t, s t + 1 ) = Q t ( s t, s t + 1 ) + k · [ R ( s t, s t + 1 ) + γ · max a t Q t ( s t, s t + 1 ) − Q t ( s t, s t + 1 ) ] was derived. Based on the improved Q function and the state transition matrices obtained under different HRT step-lengths, the optimum combinations of HRTs in An/Ae processes in any BPR system could be obtained, in terms of the ordered pair combinations of the <current state-transition state>. Model verification was performed by applying six different influent chemical oxygen demand (COD) concentrations, varying from 150 to 600 mg L −1 and influent P concentrations, varying from 12 to 30 mg L −1 . Superior and stable effluent qualities were observed with the optimal control strategies. This indicates that the proposed novel QL-based BPR model performed properly and the derived Q functions successfully realized real-time modelling, with stable optimal control strategies under fluctuant influent loads during wastewater treatment processes. Graphical abstract: Image 1 Highlights: A fluctuant influent responsive QL-based BPR optimizing control method was developed. Q t + 1 ( s t, s t + 1 ) = Q t ( s t, s t + 1 ) + k · [ R ( s t, s t + 1 ) + γ · max a t Q t ( s t, s t + 1 ) − Q t ( s t, s t + 1 ) ] was derived. State transition matrices obtained under different HRT step-lengths were developed. Ordered pair of <current state-transition state > corresponds optimal control strategy. Superior effluents achieved by optimal control strategies confirm the model validity.