Data‐aware and simulation‐driven planning of scientific workflows on IaaS clouds. (14th November 2021)

• Record Type: Journal Article
• Authors: N'Takpé, Tchimou; Edgard Gnimassoun, Jean; Oumtanaga, Souleymane; Suter, Frédéric
• Other Names: Wu Chase guestEditor.; Yildirim Tulay guestEditor.; Ivanovic Mirjana guestEditor.; Bellatreche Ladjel guestEditor.; Wyrzykowski Roman guestEditor.; Ciorba Florina M. guestEditor.
• Is Part Of: Concurrency and computation. Volume 34:Number 14(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-11-14
• Subjects: data‐intensive workflows -- IaaS cloud -- makespan reduction -- workflow scheduling; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6719
• Languages: English
• ISSNs: 1532-0626

Abstract

Abstract: The promise of an easy access to a virtually unlimited number of resources makes Infrastructure as a Service Clouds a good candidate for the execution of data‐intensive workflow applications composed of hundreds of computational tasks. Thanks to a careful execution planning, workflow management systems can build a tailored compute infrastructure by combining a set of virtual machine instances. However, these applications usually rely on files to handle dependencies between tasks. A storage space shared by all virtual machines may become a bottleneck and badly impact the application execution time. In this article, we propose an original data‐aware planning algorithm that leverages two characteristics of a family of virtual machines instances, that is, a large number of cores and a dedicated storage space on fast SSD drives, to improve data locality, hence reducing the amount of data transfers over the network during the execution of a workflow. We also propose a simulation‐driven approach to solve a cost‐performance optimization problem and correctly dimension the virtual infrastructure onto which execute a given workflow. Experiments conducted with real application workflows show the benefits of the presented algorithms. The data‐aware planning leads to a clear reduction of both execution time and volume of data transferred over the network while the simulation‐driven approach allows us to dimension the infrastructure in a reasonable time.