Multi-agent quantum-inspired deep reinforcement learning for real-time distributed generation control of 100% renewable energy systems. (March 2023)

• Record Type: Journal Article
• Authors: Liu, Dan; Wu, Yingzi; Kang, Yiqun; Yin, Linfei; Ji, Xiaotong; Cao, Xinghui; Li, Chuangzhi
• Is Part Of: Engineering applications of artificial intelligence. Volume 119(2023)
• Publication Date: 2023-03
• Subjects: Quantum technology -- Deep reinforcement learning -- Exploration and exploitation -- Real-time distributed generation control -- 100% renewable energy systems; Engineering -- Data processing -- Periodicals; Artificial intelligence -- Periodicals; Expert systems (Computer science) -- Periodicals; Ingénierie -- Informatique -- Périodiques; Intelligence artificielle -- Périodiques; Systèmes experts (Informatique) -- Périodiques; Artificial intelligence; Engineering -- Data processing; Expert systems (Computer science); Periodicals; 620.00285
• DOI: 10.1016/j.engappai.2022.105787
• Languages: English
• ISSNs: 0952-1976

Abstract

Abstract: With promoting peaking carbon emissions and achieving carbon neutrality, the real-time distributed control of the prosumers of 100% renewable energy systems (RESs) is challenging. This paper proposes multi-agent quantum-inspired deep reinforcement learning (QDRL) approaches for real-time distributed generation control of 100% RESs. Quantum-inspired operation is introduced into deep reinforcement learning (DRL) as quantum-inspired Q-learning, quantum-inspired state–action–reward-state–action, quantum-inspired deep Q-network, quantum-inspired policy gradient, quantum-inspired deep deterministic policy gradient, quantum-inspired twin-delayed deep deterministic policy gradient, quantum-inspired actor–critic, ​ quantum-inspired proximal policy optimization, and quantum-inspired soft actor–critic.​ These proposed nine QDRL approaches are compared with DRL approaches under two 100% RESs. The numeric results show that the QDRL obtains more minor carbon emissions and frequency deviations under complex 100% RESs. Moreover, the quantum states of QDRL match the uncertain states of the prosumers of 100% RESs. Besides, the exploration and exploitation of the QDRL for the real-time control problems of multi-agent systems are verified and analyzed. Highlights: Real-time active power balances of 100% renewable energy systems are considered. Value or policy-based, and actor–critic reinforcement learning methods are quantized. Generalized multi-agent quantum deep reinforcement learning methods are proposed. Higher generation control performances and smaller carbon emission are obtained. Convergence of generalized quantum deep reinforcement learning methods is analyzed.