Reducing the effect of global synchronization in delayed gradient methods for symmetric linear systems. (September 2020) - Zou, Qinmeng; Magoulès, Frédéric
Highlights: Formulation of cyclic s-dimensional gradient methods. Establishment of new properties for s-dimentional steepest descent. Comparison of parallel gradient iterative schemes. Discussion of the damped strategy and the equilibration technique. Experimental confirmation of the success of our strategy. Abstract: Compared with arithmetic operation, communication cost is often the bottleneck on modern computers, and thus should be paid increasing attention when choosing algorithms. Lagged gradient methods are known for their error tolerance and fast convergence. However, it appears that their parallel behavior is not well understood. In this paper, we explore the cyclic formulations of lagged gradient methods and s -dimensional methods for reducing global synchronizations. We provide parallel implementations for these methods and propose some new variants. A comparison is then reported for different gradient iterative schemes. To illustrate the performance, we run a number of experiments, from which we conclude that our formulations perform better than traditional methods in view of both iteration count and computing time.