PANORAMA: An approach to performance modeling and diagnosis of extreme-scale workflows. (January 2017)

• Record Type: Journal Article
• Authors: Deelman, Ewa; Carothers, Christopher; Mandal, Anirban; Tierney, Brian; Vetter, Jeffrey S; Baldin, Ilya; Castillo, Claris; Juve, Gideon; Król, Dariusz; Lynch, Vickie; Mayer, Ben; Meredith, Jeremy; Proffen, Thomas; Ruth, Paul; Ferreira da Silva, Rafael
• Is Part Of: International journal of high performance computing applications. Volume 31:Number 1(2017)
• Page Start: 4
• Page End: 18
• Publication Date: 2017-01
• Subjects: Performance modeling -- extreme scale -- scientific workflow; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/1094342015594515
• Languages: English
• ISSNs: 1094-3420

Abstract

Computational science is well established as the third pillar of scientific discovery and is on par with experimentation and theory. However, as we move closer toward the ability to execute exascale calculations and process the ensuing extreme-scale amounts of data produced by both experiments and computations alike, the complexity of managing the compute and data analysis tasks has grown beyond the capabilities of domain scientists. Thus, workflow management systems are absolutely necessary to ensure current and future scientific discoveries. A key research question for these workflow management systems concerns the performance optimization of complex calculation and data analysis tasks. The central contribution of this article is a description of the PANORAMA approach for modeling and diagnosing the run-time performance of complex scientific workflows. This approach integrates extreme-scale systems testbed experimentation, structured analytical modeling, and parallel systems simulation into a comprehensive workflow framework called Pegasus for understanding and improving the overall performance of complex scientific workflows.