Reducing the effect of global synchronization in delayed gradient methods for symmetric linear systems. (September 2020)

• Record Type: Journal Article
• Authors: Zou, Qinmeng; Magoulès, Frédéric
• Is Part Of: Advances in engineering software. Volume 147(2020)
• Publication Date: 2020-09
• Subjects: Lagged gradient methods -- Reducing synchronization -- S-dimensional gradient methods -- Cyclic gradient methods -- Parallel computing -- Asynchronous iterations; 65F10 -- 68Q85; Computer-aided engineering -- Periodicals; Engineering -- Computer programs -- Periodicals; Engineering -- Software -- Periodicals; Periodicals; 620.0028553
• DOI: 10.1016/j.advengsoft.2020.102837
• Languages: English
• ISSNs: 0965-9978

Abstract

Highlights: Formulation of cyclic s-dimensional gradient methods. Establishment of new properties for s-dimentional steepest descent. Comparison of parallel gradient iterative schemes. Discussion of the damped strategy and the equilibration technique. Experimental confirmation of the success of our strategy. Abstract: Compared with arithmetic operation, communication cost is often the bottleneck on modern computers, and thus should be paid increasing attention when choosing algorithms. Lagged gradient methods are known for their error tolerance and fast convergence. However, it appears that their parallel behavior is not well understood. In this paper, we explore the cyclic formulations of lagged gradient methods and s -dimensional methods for reducing global synchronizations. We provide parallel implementations for these methods and propose some new variants. A comparison is then reported for different gradient iterative schemes. To illustrate the performance, we run a number of experiments, from which we conclude that our formulations perform better than traditional methods in view of both iteration count and computing time.