Load balancing in MapReduce on homogeneous and heterogeneous clusters: an in-depth review. (2015)

• Record Type: Journal Article
• Authors: Kargar, Mohammad Javad; Vakili, Meysam
• Is Part Of: International journal of communication networks and distributed systems. Volume 15:Number 2/3(2015)
• Page Start: 149
• Page End: 168
• Publication Date: 2015
• Subjects: MapReduce -- Hadoop -- load balancing -- big data -- homogeneous clusters -- heterogeneous clusters -- cloud computing -- data locality -- data skew -- optimisation; Computer networks -- Periodicals; Telecommunication systems -- Periodicals; Electronic data processing -- Distributed processing -- Periodicals; 004.6
• Languages: English
• ISSNs: 1754-3916

Abstract

Numbers of various programming models have been proposed to process big data in recent years. However, MapReduce is the most famous programming model amongst cloud computing environments and includes many advantages, yet there are several challenges to deal with. Load balancing is considered as one of the most significant downsides of MapReduce which causes the increase in applications' runtime and accordingly results in less-efficiency, where there is no appropriate proposed mechanism. Although, data locality and data skew are known as two main key factors for determination of load balancing, yet it is remarkable that load balance highly depends on whether the computational clusters are homogeneous or heterogeneous. This paper examines the effectiveness of two main key factors. These are data locality and data skew on homogeneous and heterogeneous clusters. Besides, a review is conducted on a number of recent literature in the same context of load balancing improvements in Hadoop MapReduce. Finally, all investigated researches are compared with the purpose of highlighting the differences of various load balancing methods, the optimisation phase, type of clusters and the main challenges.