Demystifying asynchronous I/O Interference in HPC applications. (July 2021)

• Record Type: Journal Article
• Authors: Tseng, Shu-Mei; Nicolae, Bogdan; Cappello, Franck; Chandramowlishwaran, Aparna
• Is Part Of: International journal of high performance computing applications. Volume 35:Number 4(2021)
• Page Start: 391
• Page End: 412
• Publication Date: 2021-07
• Subjects: I/O interference -- asynchronous and concurrent I/O -- checkpointing -- HPC applications -- performance analysis; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/10943420211016511
• Languages: English
• ISSNs: 1094-3420

Abstract

With increasing complexity of HPC workflows, data management services need to perform expensive I/O operations asynchronously in the background, aiming to overlap the I/O with the application runtime. However, this may cause interference due to competition for resources: CPU, memory/network bandwidth. The advent of multi-core architectures has exacerbated this problem, as many I/O operations are issued concurrently, thereby competing not only with the application but also among themselves. Furthermore, the interference patterns can dynamically change as a response to variations in application behavior and I/O subsystems (e.g. multiple users sharing a parallel file system). Without a thorough understanding, I/O operations may perform suboptimally, potentially even worse than in the blocking case. To fill this gap, this paper investigates the causes and consequences of interference due to asynchronous I/O on HPC systems. Specifically, we focus on multi-core CPUs and memory bandwidth, isolating the interference due to each resource. Then, we perform an in-depth study to explain the interplay and contention in a variety of resource sharing scenarios such as varying priority and number of background I/O threads and different I/O strategies: sendfile, read/write, mmap/write underlining trade-offs. The insights from this study are important both to enable guided optimizations of existing background I/O, as well as to open new opportunities to design advanced asynchronous I/O strategies.