Enhancing hybrid parallel file system through performance and space-aware data layout. (November 2016)

• Record Type: Journal Article
• Authors: He, Shuibing; Liu, Yan; Wang, Yang; Sun, Xian-He; Huang, Chuanhe
• Is Part Of: International journal of high performance computing applications. Volume 30:Number 4(2016:Winter)
• Page Start: 396
• Page End: 410
• Publication Date: 2016-11
• Subjects: Parallel I/O system -- parallel file system -- data layout -- solid-state drive -- hybrid I/O system; High performance computing -- Periodicals; Supercomputers -- Periodicals; 004.1105
• DOI: 10.1177/1094342016631610
• Languages: English
• ISSNs: 1094-3420

Abstract

Hybrid parallel file systems (PFSs), which consist of solid-state drive servers (SServer) and hard disk drive servers (HServer), have recently attracted growing attention. Compared to a traditional HServer, an SServer consistently provides improved storage performance but lacks storage space. However, most current data layout schemes do not consider the differences in performance and space between heterogeneous servers and may significantly degrade the performance of the hybrid PFSs. In this article, we propose performance and space-aware (PSA) scheme, a novel data layout scheme, which maximizes the hybrid PFSs' performance by applying adaptive varied-size file stripes. PSA dispatches data on heterogeneous file servers not only based on storage performance but also storage space. We have implemented PSA within OrangeFS, a popular PFS in the high-performance computing domain. Our extensive experiments with representative benchmarks, including IOR, HPIO, MPI-TILE-IO, and BTIO, show that PSA provides superior I/O throughput than the default and performance-aware file data layout schemes.