Cogset: a high performance MapReduce engine. (24th May 2012)

• Record Type: Journal Article
• Authors: Valvåg, S.V.; Johansen, Dag; Kvalnes, Åge; Fedak, Gilles; Li, Keqiu; Lin, Kai
• Is Part Of: Concurrency and computation. Volume 25:Number 1(2013:Jan.)
• Page Start: 2
• Page End: 23
• Publication Date: 2012-05-24
• Subjects: Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.2827
• Languages: English
• ISSNs: 1532-0626

Abstract

<abstract abstract-type="main" id="cpe2827-abs-0001"> <title>SUMMARY</title> <p>Cogset is a generic and efficient engine for reliable storage and parallel processing of distributed data sets. It supports a number of high‐level programming interfaces, including a MapReduce interface compatible with Hadoop. In this paper, we present Cogset's architecture and evaluate its performance as a MapReduce engine, comparing it with Hadoop. Our results show that Cogset generally outperforms Hadoop by a significant margin. We investigate the underlying causes of this difference in performance and demonstrate some relatively minor modifications that markedly improve Hadoop's performance, closing some of the gap. Copyright © 2012 John Wiley &amp; Sons, Ltd.</p> </abstract>