A cost‐efficient resemblance detection scheme for post‐deduplication delta compression in backup systems. (9th August 2021) - Wang, Chunzhi; Fu, Yanlin; Yan, Junyi; Wu, Xinyun; Zhang, Yucheng; Xia, Huiling; Yuan, Ye
Summary: Delta compression, which is efficient in removing repeated string among similar chunks, can be used as a complement to data deduplication in backup storage for extra space savings. The process of detecting similar candidates to use as the base for delta compression is called resemblance detection. Several indexes are required for resemblance detection. Maintaining them in RAM would limit the system scalability and increase system cost. Storing them on the disk suffers from low throughput due to poor random I/O performance of the disk. In this article, we present the history‐aware resemblance detection (HARD), a cost‐efficient resemblance detection approach that captures most of the similar chunks with a limited memory footprint. HARD is based on the observation that, for chunks in a backup, most of their similar chunks can be found in the most recent backups. HARD thus only indexes super‐features in the most recent backups for resemblance detection to reduce the memory footprint of resemblance indexes while captures most of the potential similar chunks for delta compression. Experimental results based on three real‐world datasets show that HARD achieves higher compression than the state‐of‐the‐art approach.