P-DOT: a model of computation for big data. Issue 3 (3rd May 2016)

• Record Type: Journal Article
• Authors: Luo, Tao; Liao, Yin; Chen, Guoliang; Zhang, Yunquan
• Is Part Of: International journal of parallel, emergent and distributed systems. Volume 31:Issue 3(2016)
• Page Start: 233
• Page End: 253
• Publication Date: 2016-05-03
• Subjects: big data -- computational model -- distributed system -- complexity analysis; Parallel computers -- Periodicals; Electronic data processing -- Distributed processing -- Periodicals; Computer algorithms -- Periodicals; 004.35
• DOI: 10.1080/17445760.2015.1016515
• Languages: English
• ISSNs: 1744-5760

Abstract

Abstract : In response to the high demand of big data analytics, several programming models on large and distributed cluster systems have been proposed and implemented, such as MapReduce, Dryad and Pregel. However, compared with high performance computing areas, the basis and principles of computation and communication behaviour of big data analytics is not well studied. In this paper, we review the current big data computational model DOT and DOT Advanced, and propose a more general and practical model p-DOT (p-phases DOT). p-DOT is not a simple extension, but with profound significance: for general aspects, any big data analytics job execution expressed in DOT model or bulk synchronous parallel model can be represented by it; for practical aspects, it considers I/O behaviour to evaluate performance overhead. Moreover, we provide a cost function of p-DOT implying that the optimal number of machines is near-linear to the square root of input size for a fixed algorithm and workload, and certify that the processing paradigm of p-DOT is scalable and fault-tolerant. Finally, we demonstrate the effectiveness of the model through several experiments.