Efficient erasure-coded data updates based on file class predictions and hybrid writes. (December 2022)

• Record Type: Journal Article
• Authors: Wei, Bing; Wu, Jigang; Su, Xiaosong; Huang, Qiang; Liu, Yujun; Zhang, Fuhao
• Is Part Of: Computers & electrical engineering. Volume 104:Part B(2022)
• Publication Date: 2022-12
• Subjects: Erasure coding -- Efficient partial writes -- Machine learning -- File classifications -- Data delta-based hybrid writes; Computer engineering -- Periodicals; Electrical engineering -- Periodicals; Electrical engineering -- Data processing -- Periodicals; Ordinateurs -- Conception et construction -- Périodiques; Électrotechnique -- Périodiques; Électrotechnique -- Informatique -- Périodiques; Computer engineering; Electrical engineering; Electrical engineering -- Data processing; Periodicals; Electronic journals; 621.302854
• DOI: 10.1016/j.compeleceng.2022.108441
• Languages: English
• ISSNs: 0045-7906

Abstract

Abstract: A small update write can lead to a partial write to an erasure coding group in erasure-coded storage systems, resulting in a time-consuming write-after-read. This paper presents a data delta and logging based writing approach, named DETOG, to achieve fast partial writes. DETOG classifies files into non-write-only and write-only using machine learning, then adopts different write schemes to perform partial writes according to file classes. For non-write-only files, DETOG uses data deltas that are the differences between latest data values and original data values, instead of parity deltas to reconstruct the failed data. For non-write-only files, DETOG performs in-place data updates and log-based parity updates, so as to balance the overhead of data updates and reads. We implement DETOG on the newly designed prototype storage system to perform performance evaluation. The results of extensive experiments on real-world traces showed that DETOG significantly improved system throughput compared with state-of-the-art methods. Graphical abstract: Highlights: We aim to minimize the execution time of partial writes. We use file class predictions and hybrid writes to perform partial writes. We perform partial writes for different classes of files. We use data deltas instead of parity deltas to reconstruct the failed data.