Deadline scheduling algorithm for sustainable computing in Hadoop environment. Issue 76 (July 2018)

• Record Type: Journal Article
• Authors: Varga, Mihai; Petrescu-Nita, Alina; Pop, Florin
• Is Part Of: Computers & security. Issue 76(2018)
• Page Start: 354
• Page End: 366
• Publication Date: 2018-07
• Subjects: Scheduling -- Distributed systems -- Deadline scheduling -- Hadoop -- Energy efficiency -- Sustainable computing; Computer security -- Periodicals; Electronic data processing departments -- Security measures -- Periodicals; 005.805
• DOI: 10.1016/j.cose.2017.12.014
• Languages: English
• ISSNs: 0167-4048

Abstract

Highlights: Generic scheduling algorithm with deadline constraints. Cost model that estimates the remaining work load. A model that properly prioritizes jobs according to their upcoming deadlines. Support for sustainable computing considering real-time processing. Performance evaluation in Amazon EC2. Abstract: Cloud computing is popular choice for processing and analyzing large amounts of data. Organizations can easily manage and deploy powerful clusters that run different software environments and enable distributed processing. Scheduling is an important part of distributed computing that allows users to leverage the available resources for a faster computation time. In this paper we propose a generic scheduling algorithm that takes deadline constraints into consideration. We develop a cost model that estimates the remaining work load which allows the scheduler to properly prioritize jobs according to their upcoming deadlines. The cost model works with generic abstract resources requests such as virtual cores, memory and containers and determines the remaining running time based on the completed tasks. We validate the cost model and measure the performance of the scheduler by running several experiments on a cluster on Amazon EC2 and our algorithm performs as expected under different scenarios.