Scheduling DAG-based workflows on single cloud instances: High-performance and cost effectiveness with a static scheduler. (January 2017)

• Record Type: Journal Article
• Authors: Taufer, Michela; Rosenberg, Arnold L.
• Is Part Of: International journal of high performance computing applications. Volume 31:Number 1(2017)
• Page Start: 19
• Page End: 31
• Publication Date: 2017-01
• Subjects: Area-oriented -- DAG-scheduler -- locally optimal -- DAG-scheduler -- Amazon Elastic Computing Cloud; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/1094342015594518
• Languages: English
• ISSNs: 1094-3420

Abstract

The problem of achieving high-performance cost-effectively in cloud computing is challenging when workflows have Directed Acyclic Graph (DAG)-structured inter-task dependencies. We study this problem within single cloud instances and provide empirical evidence that the static Area-Oriented DAG-Scheduling (AO) paradigm, which predetermines the order for executing a DAG's tasks, provides both high performance and cost effectiveness. AO produces schedules in a platform-oblivious manner; it ignores the performance characteristics of the platform's resources and focuses only on the dependency structure of the workflow. Specifically, AO's schedules strive to enhance the rate of rendering tasks eligible for execution. Using an archive of diverse DAG-structured workflows, we experimentally compare AO with a variety of competing DAG-schedulers: (a) the static locally optimal DAG-scheduler (LO), which, like AO, is static and platform-oblivious but chooses its DAG-ordering based on tasks' outdegrees; and (b) five dynamic versions of static schedulers (including AO and LO), each of which can violate its parent static scheduler's prescribed task orders to avoid stalling. Our results provide evidence of AO's supremacy as compared with LO and its essential equivalence to dynamic-AO: neither competitor yields higher performance at an lower cost than AO does. Two aspects of these results are notable. Firstly, AO is platform-oblivious, whereas dynamic-AO is intensely platform-sensitive; one would expect platform sensitivity to enhance performance. Secondly, AO outperforms LO by an order of magnitude, together with lower costs; one would not expect such a performance gap.