Optimizing MapReduce for energy efficiency. (26th June 2018)

• Record Type: Journal Article
• Authors: Tiwari, Nidhi; Bellur, Umesh; Sarkar, Santonu; Indrawan, Maria
• Is Part Of: Software, practice & experience. Volume 48:Number 9(2018)
• Page Start: 1660
• Page End: 1687
• Publication Date: 2018-06-26
• Subjects: distributed computing -- energy efficiency -- energy model -- MapReduce systems -- optimal configuration -- performance model; Computer software -- Periodicals; Computer programming -- Periodicals; Computer programs -- Periodicals; 005.3
• DOI: 10.1002/spe.2599
• Languages: English
• ISSNs: 0038-0644

Abstract

Summary: The efficient use of energy is essential to address concerns of cost and sustainability. Many data centers contain MapReduce clusters to process Big Data applications. A large number of machines and fault tolerance capabilities make MapReduce clusters energy inefficient. In this paper, we present a Configurator based on performance and energy models to improve the energy efficiency of MapReduce systems. Our solution is novel as it takes into account the dependence of the performance and energy consumption of a cluster on MapReduce parameters. While this dependence is known, we are the first to model it and design a Configurator to optimize these parameter settings for maximizing the energy efficiency of MapReduce systems. Our empirical evaluations show that the Configurator can result in up to 50% improvement in the energy efficiency of typical MapReduce applications in two architecturally different clusters.