A cost‐efficient resemblance detection scheme for post‐deduplication delta compression in backup systems. (9th August 2021)

• Record Type: Journal Article
• Authors: Wang, Chunzhi; Fu, Yanlin; Yan, Junyi; Wu, Xinyun; Zhang, Yucheng; Xia, Huiling; Yuan, Ye
• Is Part Of: Concurrency and computation. Volume 34:Number 3(2022)
• Page Start: n/a
• Page End: n/a
• Publication Date: 2021-08-09
• Subjects: backup storage -- post‐deduplication delta compression -- resemblance detection; Parallel processing (Electronic computers) -- Periodicals; Parallel computers -- Periodicals; 004.35
• DOI: 10.1002/cpe.6558
• Languages: English
• ISSNs: 1532-0626

Abstract

Summary: Delta compression, which is efficient in removing repeated string among similar chunks, can be used as a complement to data deduplication in backup storage for extra space savings. The process of detecting similar candidates to use as the base for delta compression is called resemblance detection. Several indexes are required for resemblance detection. Maintaining them in RAM would limit the system scalability and increase system cost. Storing them on the disk suffers from low throughput due to poor random I/O performance of the disk. In this article, we present the history‐aware resemblance detection (HARD), a cost‐efficient resemblance detection approach that captures most of the similar chunks with a limited memory footprint. HARD is based on the observation that, for chunks in a backup, most of their similar chunks can be found in the most recent backups. HARD thus only indexes super‐features in the most recent backups for resemblance detection to reduce the memory footprint of resemblance indexes while captures most of the potential similar chunks for delta compression. Experimental results based on three real‐world datasets show that HARD achieves higher compression than the state‐of‐the‐art approach.