Reinforcement learning approach for robustness analysis of complex networks with incomplete information. (March 2021)

• Record Type: Journal Article
• Authors: Tian, Meng; Dong, Zhengcheng; Wang, Xianpei
• Is Part Of: Chaos, solitons and fractals. Volume 144(2021)
• Publication Date: 2021-03
• Subjects: Robustness -- Complex networks -- Sequential attacks -- Incomplete information -- Reinforcement learning -- Deep learning; Chaotic behavior in systems -- Periodicals; Solitons -- Periodicals; Fractals -- Periodicals; Chaotic behavior in systems; Fractals; Solitons; Periodicals; 003.7
• DOI: 10.1016/j.chaos.2020.110643
• Languages: English
• ISSNs: 0960-0779

Abstract

Highlights: Robustness of complex networks is analyzed by reinforcement learning. Large state space of complex networks is addressed by Deep Q-network algorithm. RL-based sequential attacks are affected by load distributions and network structures. RL-based sequential attacks perform better with more observed and attacked nodes. Abstract: Network robustness against sequential attacks is significant for complex networks. However, it is generally assumed that complete information of complex networks is obtained and arbitrary nodes can be removed in previous researches. In this paper, a sequential attack in complex networks is modeled as a partial observable Markov decision process (POMDP). Then a reinforcement learning (RL) approach for POMDP is proposed to analyze dynamical robustness of complex networks under sequential attacks, when information of networks is incomplete. According to this approach, an agent can learn to take action by exploiting experiences. To solve the problem of large state space in complex networks, deep Q -network algorithm is used to identify most damaging sequential attacks, as deep neural networks can build up progressively abstract representations of state space of complex networks. The performances of proposed approach are analyzed on scale-free networks and small-world networks. According to the numerical simulations, it is found that the RL-based sequential attacks perform better when load distributions are more heterogeneous and local connections are more significant. Furthermore, it is shown that increasing the proportions of observed and attacked nodes improves the performance of RL-based sequential attacks. Finally, the results are verified on the IEEE 300-bus system and the simulation results highlight the damages caused by RL-based sequential attacks.