A2Cloud‐RF: A random forest based statistical framework to guide resource selection for high‐performance scientific computing on the cloud. (22nd July 2020)

• Record Type: Journal Article
• Authors: Samuel, David; Khan, Syeduzzaman; Balos, Cody J.; Abuelhaj, Zachariah; Dutoi, Anthony D.; Kari, Chadi; Mueller, David; Pallipuram, Vivek K.
• Is Part Of: Concurrency and computation. Volume 32:Number 24(2020)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2020-07-22
• Subjects: capacity planning -- cloud computing -- cloud framework -- scientific computing; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.5942
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: This article proposes a random‐forest based A2Cloud framework to match scientific applications with Cloud providers and their instances for high performance. The framework leverages four engines for this task: PERF engine, Cloud trace engine, A2Cloud‐ext engine, and the random forest classifier (RFC) engine. The PERF engine profiles the application to obtain performance characteristics, including the number of single‐precision (SP) floating‐point operations (FLOPs), double‐precision (DP) FLOPs, x87 operations, memory accesses, and disk accesses. The Cloud trace engine obtains the corresponding performance characteristics of the selected Cloud instances including: SP floating point operations per second (FLOPS), DP FLOPS, x87 operations per second, memory bandwidth, and disk bandwidth. The A2Cloud‐ext engine uses the application and Cloud instance characteristics to generate objective scores that represent the application‐to‐Cloud match. The RFC engine uses these objective scores to generate two types of random forests to assist users with rapid analysis: application‐specific random forests (ARF) and application‐class based random forests. The ARF consider only the input application's characteristics to generate a random forest and provide numerical ratings to the selected Cloud instances. To generate the application‐class based random forests, the RFC engine downloads the application profiles and scores of previously tested applications that perform similar to the input application. Using these data, the RFC engine creates a random forest for instance recommendation. We exhaustively test this framework using eight real‐world applications across 12 instances from different Cloud providers. Our tests show significant statistical agreement between the instance ratings given by the framework and the ratings obtained via actual Cloud executions.