Decentralised federated learning with adaptive partial gradient aggregation. Issue 3 (9th September 2020)

• Record Type: Journal Article
• Authors: Jiang, Jingyan; Hu, Liang
• Is Part Of: CAAI transactions on intelligence technology. Volume 5:Issue 3(2020)
• Page Start: 230
• Page End: 236
• Publication Date: 2020-09-09
• Subjects: learning (artificial intelligence) -- gradient methods; communication frequency -- parameter server design -- nodes‐to‐server bandwidths -- stochastic gradient descent training -- end‐to‐end training -- real‐world federated learning scenarios -- adaptive partial gradient aggregation method -- gradient partial level decentralised -- partial gradient exchange mechanism -- node‐to‐node bandwidth -- communication time -- adaptive model -- training time -- decentralised federated learning -- machine learning model -- geo‐distributed workers -- inherently communication -- communication efficiency -- conventional federated learning algorithms; Artificial intelligence -- Periodicals; Computer science -- Periodicals; Artificial intelligence; Computer science; Electronic journals; Periodicals; 006.305
• DOI: 10.1049/trit.2020.0082
• Languages: English
• ISSNs: 2468-6557

Abstract

Abstract : Federated learning aims to collaboratively train a machine learning model with possibly geo‐distributed workers, which is inherently communication constrained. To achieve communication efficiency, the conventional federated learning algorithms allow the worker to decrease the communication frequency by training the model locally for multiple times. Conventional federated learning architecture, inherited from the parameter server design, relies on highly centralised topologies and large nodes‐to‐server bandwidths, and convergence property relies on the stochastic gradient descent training in local, which usually causes the large end‐to‐end training latency in real‐world federated learning scenarios. Thus, in this study, the authors propose the adaptive partial gradient aggregation method, a gradient partial level decentralised federated learning, to tackle this problem. In FedPGA, they propose a partial gradient exchange mechanism that makes full use of node‐to‐node bandwidth for speeding up the communication time. Besides, an adaptive model updating method further reduces the convergence rate by adaptive increasing the step size of the stable direction of gradient descent. The experimental results on various datasets demonstrate that the training time is reduced up to 14 × compared to baselines without accuracy degrade.