Novel heuristic speculative execution strategies in heterogeneous distributed environments. (February 2016)

• Record Type: Journal Article
• Authors: Huang, Xin; Zhang, Longxin; Li, Renfa; Wan, Lanjun; Li, Keqin
• Is Part Of: Computers & electrical engineering. Volume 50(2016)
• Page Start: 166
• Page End: 179
• Publication Date: 2016-02
• Subjects: Cloud computing -- Dynamic loading -- Hadoop -- MapReduce -- Speculative execution; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2015.06.013
• Languages: English
• ISSNs: 0045-7906

Abstract

Graphical abstract: Highlights: A novel speculative strategy of remaining time estimation is presented. An extensional maximum cost performance is developed. The system load is considered while estimating the remaining time. The proposed Hadoop-ERUL works more precisely and rapidly. Abstract: MapReduce is a promising distributed computing platform for large-scale data processing applications. Hadoop MapReduce has been considered as one of the most extensively used open-source implementations of MapReduce frameworks for its flexible customization and convenient usage. Despite these advantages, a relatively slow running task called straggler task impedes job progress. In this study, two novel speculative strategies, namely, Estimate Remaining time Using Linear relationship model (ERUL) and extensional Maximum Cost Performance (exMCP), are developed to improve the estimation of the remaining time of a task. ERUL is a dynamic system load-aware strategy; using this strategy, we can overcome some drawbacks of the Longest Approximate Time to End (LATE) that misleads speculative execution in some cases. In exMCP, different slot values are considered. Extensive experiments show that ERUL and exMCP are applied to accurately estimate the remaining execution times of running tasks and reduce the running time of a job.