In‐network block repairing for erasure coding storage systems. (26th July 2019)

• Record Type: Journal Article
• Authors: Xia, Junxu; Guo, Deke; Cheng, Geyao
• Other Names: Li Gang guestEditor.; Batten Lynn guestEditor.; Foschini Luca guestEditor.; Kim Hyunbum guestEditor.; Dong Fang guestEditor.; Wu Chenshu guestEditor.; Gao Shangce guestEditor.
• Is Part Of: Concurrency and computation. Volume 31:Number 24(2019)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2019-07-26
• Subjects: data center -- distributed system -- erasure code -- in‐network aggregation; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.5432
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: In the erasure coding storage system, it is necessary to extract multiple data blocks from other remaining storage nodes to a new node when a storage node fails, which repairs the failed data block satisfactorily. However, this would incur the incast problem at this new node. The existing solutions for the repair process in the incast problem mainly rely on path planning and resource allocation. Although these solutions improve the performance of repairing the failed data blocks, they still waste a large amount of storage and bandwidth resources unavoidably. In this paper, we propose the incast problem to be resolved economically via the in‐network aggregation. Specifically, we assume that the switches in data centers have certain data processing capabilities and can aggregate data flows efficiently. Thereafter, we propose a set of in‐network methods to repair a failed data block in the erasure coding storage systems, taking the fat‐tree data center as an example. Thus, the incast problem can be solved effectively during the data transmission process. Compared with the prior methods, our approach effectively avoids the overhead of extra path computing, as well as significantly reduces the link cost of repairing data blocks, while promising similar or faster repair speed.