An Efficient Message Filtering Strategy Based on Asynchronous ADMM with L1 Regularization. (August 2019)

• Record Type: Journal Article
• Authors: Zhang, Jiafeng
• Is Part Of: Journal of physics. Volume 1284(2019)
• Publication Date: 2019-08
• Subjects: Physics -- Congresses; 530.5
• DOI: 10.1088/1742-6596/1284/1/012066
• Languages: English
• ISSNs: 1742-6588

Abstract

Abstract: With the increment of data scale, distributed machine learning has received more and more attention. However, as the data grows, the dimension of the dataset will increase rapidly, which leads to the increment of the communication traffic in the distributed computing cluster and decreases the performance of the distributed algorithms. This paper proposes a message filtering strategy based on asynchronous alternating direction method of multipliers (ADMM), which can effectively reduce the communication time of the algorithm while ensuring the convergence of the algorithm. In this paper, a soft threshold filtering strategy based on L1 regularization is proposed to filter the parameter of master node, and a gradient truncation filtering strategy is proposed to filter the parameter of slave node. Besides, we update the algorithm asynchronously to reduce the waiting time of the master node. Experiments on large-scale sparse data show that our algorithm can effectively reduce the traffic of messages and make the algorithm reach convergence in a shorter time.