How to parallelize Rsync. (November 2020)

• Record Type: Journal Article
• Authors: Zhang, Changjian; Qi, Deyu
• Is Part Of: Journal of physics. Volume 1684(2020)
• Publication Date: 2020-11
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/1684/1/012080
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: With the advent of the data age, data backup has become the focus of academic research, especially the incremental backup technology based on Rsync. However, the current Rsync does not support parallel work because it needs to keep a long connection and needs multiple segments of communication to complete a single synchronization process. Therefore, this paper uses the Kafka framework to decouple the synchronization process of the Rsync algorithm, optimizes the synchronization process based on shadow data, and eventually endows the Rsync algorithm with the ability of parallel work. Experimental results show that the proposed method can effectively improve the concurrency of Rsync. This method will be beneficial to the performance optimization of the server in the backup system.