A codesign framework for online data analysis and reduction. (26th August 2021)
- Record Type:
- Journal Article
- Title:
- A codesign framework for online data analysis and reduction. (26th August 2021)
- Main Title:
- A codesign framework for online data analysis and reduction
- Authors:
- Mehta, Kshitij
Allen, Bryce
Wolf, Matthew
Logan, Jeremy
Suchyta, Eric
Singhal, Swati
Choi, Jong Y.
Takahashi, Keichi
Huck, Kevin
Yakushin, Igor
Sussman, Alan
Munson, Todd
Foster, Ian
Klasky, Scott - Other Names:
- Wu Chase guestEditor.
Yildirim Tulay guestEditor.
Ivanovic Mirjana guestEditor.
Bellatreche Ladjel guestEditor.
Wyrzykowski Roman guestEditor.
Ciorba Florina M. guestEditor. - Abstract:
- Abstract: Science applications preparing for the exascale era are increasingly exploring in situ computations comprising of simulation‐analysis‐reduction pipelines coupled in‐memory. Efficient composition and execution of such complex pipelines for a target platform is a codesign process that evaluates the impact and tradeoffs of various application‐ and system‐specific parameters. In this article, we describe a toolset for automating performance studies of composed HPC applications that perform online data reduction and analysis. We describe Cheetah, a new framework for composing parametric studies on coupled applications, and Savanna, a runtime engine for orchestrating and executing campaigns of codesign experiments. This toolset facilitates understanding the impact of various factors such as process placement, synchronicity of algorithms, and storage versus compute requirements for online analysis of large data. Ultimately, we aim to create a catalog of performance results that can help scientists understand tradeoffs when designing next‐generation simulations that make use of online processing techniques. We illustrate the design of Cheetah and Savanna, and present application examples that use this framework to conduct codesign studies on small clusters as well as leadership class supercomputers.
- Is Part Of:
- Concurrency and computation. Volume 34:Number 14(2022)
- Journal:
- Concurrency and computation
- Issue:
- Volume 34:Number 14(2022)
- Issue Display:
- Volume 34, Issue 14 (2022)
- Year:
- 2022
- Volume:
- 34
- Issue:
- 14
- Issue Sort Value:
- 2022-0034-0014-0000
- Page Start:
- n/a
- Page End:
- n/a
- Publication Date:
- 2021-08-26
- Subjects:
- Cheetah -- codesign -- CODAR -- exascale -- in situ -- online -- reduction -- Savanna -- workflows
Parallel processing (Electronic computers) -- Periodicals
Parallel computers -- Periodicals
004.35 - Journal URLs:
- http://onlinelibrary.wiley.com/ ↗
- DOI:
- 10.1002/cpe.6519 ↗
- Languages:
- English
- ISSNs:
- 1532-0626
- Deposit Type:
- Legaldeposit
- View Content:
- Available online (eLD content is only available in our Reading Rooms) ↗
- Physical Locations:
- British Library DSC - 3405.622000
British Library DSC - BLDSS-3PM
British Library STI - ELD Digital store - Ingest File:
- 21570.xml