Resource management for deadline constrained MapReduce jobs for minimising energy consumption. (2018)

• Record Type: Journal Article
• Authors: Gregory, Adam; Majumdar, Shikharesh
• Is Part Of: International journal of big data intelligence. Volume 5:Number 4(2018)
• Page Start: 270
• Page End: 287
• Publication Date: 2018
• Subjects: resource management on clouds -- MapReduce with deadlines -- constraint programming -- energy management -- big data analytics -- job turnaround time -- big data -- service level agreement; Big data -- Periodicals; 005.705
• Languages: English
• ISSNs: 2053-1389

Abstract

Cloud computing has emerged as one of the leading platforms for processing large-scale data intensive applications. Such applications are executed in large clusters and data centres which require a substantial amount of energy. Energy consumption within data centres accounts for a considerable fraction of costs and is a significant contributor to global greenhouse gas emissions. Therefore, minimising energy consumption in data centres is a critical concern for data centre operators, cluster owners, and cloud service providers. In this paper, we devise a novel energy aware MapReduce resource manager for an open system, called EAMR-RM, that can effectively perform matchmaking and scheduling of MapReduce jobs each of which is characterised by a service level agreement (SLA) for performance that includes a client specified earliest start time, execution time, and a deadline with the objective of minimising data centre energy consumption. Performance analysis demonstrates that for a range of system and workload parameters experimented with the proposed technique can effectively satisfy SLA requirements while achieving up to a 45% reduction in energy consumption compared to approaches which do not consider energy in resource management decisions.