Context‐aware scheduling in MapReduce: a compact review. (12th October 2015)

• Record Type: Journal Article
• Authors: Idris, Muhammad; Hussain, Shujaat; Ali, Maqbool; Abdulali, Arsen; Siddiqi, Muhammad Hameed; Kang, Byeong Ho; Lee, Sungyoung
• Is Part Of: Concurrency and computation. Volume 27:Number 17(2015:Dec.)
• Page Start: 5332
• Page End: 5349
• Publication Date: 2015-10-12
• Subjects: scheduling -- task scheduling -- job scheduling -- data‐intensive computing -- big data -- cloud; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.3578
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: It is a fact that the attention of research community in computer science, business executives, and decision makers is drastically drawn by big data. As the volume of data becomes bigger, it needs performance‐oriented data‐intensive processing frameworks such as MapReduce, which can scale computation on large commodity clusters. Hadoop MapReduce processes data in Hadoop Distributed File System as jobs scheduled according to YARN fair scheduler and capacity scheduler. However, with advancement and dynamic changes in hardware and operating environments, the performance of clusters is greatly affected. Various efforts in literature have been made to address the issues of heterogeneity (i.e., clusters consisting of virtual machines and machines with different hardware), network communication, data locality, better resource utilization, and run‐time scheduling. In this paper, we present a survey to discuss various research efforts made so far to improve Hadoop MapReduce scheduling. We classify scheduling algorithms and techniques proposed in the literature so far based on their addressing areas and present a taxonomy. Furthermore, we also discuss various aspects of open issues and challenges in the scheduling of MapReduce to improve its performance. Copyright © 2015 John Wiley & Sons, Ltd.