Community detection in dynamic networks with spark. (2018)

• Record Type: Journal Article
• Authors: Piyasinghe, Priyangika R.; Chang, J. Morris
• Is Part Of: International journal of data science. Volume 3:Number 3(2018)
• Page Start: 236
• Page End: 254
• Publication Date: 2018
• Subjects: dynamic networks -- community detection -- Girvan-Newman algorithm -- large graphs -- spark; Big data -- Periodicals; Data mining -- Periodicals; Information storage and retrieval systems -- Periodicals; Decision making -- Data processing -- Periodicals; 005.7
• Languages: English
• ISSNs: 2053-0811

Abstract

Detecting the evolution of communities within dynamically changing networks is important to understand the latent structure of complex large graphs. In this paper, we present an algorithm to detect real-time communities in dynamically changing networks. We demonstrate the proposed methodology through a case study in peer-to-peer (P2P) botnet detection which is one of the major threats to network security for serving as the infrastructure that is responsible for various cyber crimes. Our method considers online community structure from time to time and improves efficiency by maintaining the same level of accuracy of community detection over time. Experimental evaluation on Apache Spark implementation of the method showed that the execution time improves over dynamic version of Girvan-Newman community detection algorithm while having a higher accuracy level.