Hadoop framework: impact of data organization on performance. (23rd May 2011)

• Record Type: Journal Article
• Authors: Tan, Yu Shyang; Tan, Jiaqi; Chng, Eng Siong; Lee, Bu‐Sung; Li, Jiaming; Date, Susumu; Chak, Hui Ping; Xiao, Xiong; Narishige, Atsushi; Sicilia, Miguel‐Angel
• Is Part Of: Software, practice & experience. Volume 43:Number 11(2013)
• Page Start: 1241
• Page End: 1260
• Publication Date: 2011-05-23
• Subjects: Computer software -- Periodicals; Computer programming -- Periodicals; Computer programs -- Periodicals; 005.3
• DOI: 10.1002/spe.1082
• Languages: English
• ISSNs: 0038-0644

Abstract

<abstract abstract-type="main" xml:lang="en" id="spe1082-abs-0001"> <title>SUMMARY</title> <p id="spe1082-para-0001">Hadoop, based on the popular MapReduce framework, is an open‐source distributed computing framework that has been gaining much popularity and usage. It aims to allow programmers to focus on building applications that deals with processing large amount of data, without having to handle other issues when performing parallel computations. However, tuning the performance of Hadoop applications is not an easy task due to the level of abstraction of the framework. In this paper, we present three case studies and some of the challenges and issues that are to be considered in performance tuning when running applications in Hadoop. The focus is mainly on the impact of input data on Hadoop's performance and how they can be tuned. Copyright © 2011 John Wiley &amp; Sons, Ltd.</p> </abstract>