Multi-agent quantum-inspired deep reinforcement learning for real-time distributed generation control of 100% renewable energy systems. (March 2023) - Liu, Dan; Wu, Yingzi; Kang, Yiqun; Yin, Linfei; Ji, Xiaotong; Cao, Xinghui; Li, Chuangzhi
Abstract: With promoting peaking carbon emissions and achieving carbon neutrality, the real-time distributed control of the prosumers of 100% renewable energy systems (RESs) is challenging. This paper proposes multi-agent quantum-inspired deep reinforcement learning (QDRL) approaches for real-time distributed generation control of 100% RESs. Quantum-inspired operation is introduced into deep reinforcement learning (DRL) as quantum-inspired Q-learning, quantum-inspired state–action–reward-state–action, quantum-inspired deep Q-network, quantum-inspired policy gradient, quantum-inspired deep deterministic policy gradient, quantum-inspired twin-delayed deep deterministic policy gradient, quantum-inspired actor–critic, ​ quantum-inspired proximal policy optimization, and quantum-inspired soft actor–critic.​ These proposed nine QDRL approaches are compared with DRL approaches under two 100% RESs. The numeric results show that the QDRL obtains more minor carbon emissions and frequency deviations under complex 100% RESs. Moreover, the quantum states of QDRL match the uncertain states of the prosumers of 100% RESs. Besides, the exploration and exploitation of the QDRL for the real-time control problems of multi-agent systems are verified and analyzed. Highlights: Real-time active power balances of 100% renewable energy systems are considered. Value or policy-based, and actor–critic reinforcement learning methods are quantized. Generalized multi-agent quantum deep reinforcement learning methods are proposed. Higher generation control performances and smaller carbon emission are obtained. Convergence of generalized quantum deep reinforcement learning methods is analyzed.