A MapReduce‐based parallel K‐means clustering for large‐scale CIM data verification. (28th August 2015)

• Record Type: Journal Article
• Authors: Deng, Chuang; Liu, Yang; Xu, Lixiong; Yang, Jie; Liu, Junyong; Li, Siguang; Li, Maozhen
• Other Names: Frincu Mark guestEditor.; Bósa Károly guestEditor.; Rong Chunming guestEditor.; Liu Lu guestEditor.; Chen Guolong guestEditor.
• Is Part Of: Concurrency and computation. Volume 28:Number 11(2016)
• Page Start: 3096
• Page End: 3114
• Publication Date: 2015-08-28
• Subjects: CIM verification -- stochastic sampling -- clustering -- MapReduce -- load balancing; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.3580
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: The Common Information Model (CIM) has been heavily used in electric power grids for data exchange among a number of auxiliary systems such as communication systems, monitoring systems, and marketing systems. With a rapid deployment of digitalized devices in electric power networks, the volume of data continuously grows, which makes verification of CIM data a challenging issue. This paper presents a parallel K‐means clustering algorithm for large‐scale CIM data verification. The parallel K‐means builds on the MapReduce computing model which has been widely taken up by the community in dealing with data‐intensive applications. A genetic algorithm‐based load‐balancing scheme is designed to balance the workloads among the heterogeneous computing nodes for a further improvement in computation efficiency. The performance of the parallel K‐means is initially evaluated in a small‐scale in‐house MapReduce cluster and subsequently evaluated in a commercial cloud computing platform. Finally, the parallel K‐means is evaluated in large‐scale simulated MapReduce environments. Both the experimental and simulation results show that the parallel K‐means reduces the CIM data‐verification time significantly compared with the sequential K‐means clustering, while generating a high level of precision in data verification. Copyright © 2015 John Wiley & Sons, Ltd.