Print Email Facebook Twitter Federated Learning With Heterogeneity-Aware Probabilistic Synchronous Parallel on Edge Title Federated Learning With Heterogeneity-Aware Probabilistic Synchronous Parallel on Edge Author Zhao, Jianxin (Beijing Institute of Technology) Han, Rui (Beijing Institute of Technology) Yang, Yongkai (Beijing Engineering Research Center of Civil Aviation Big Data; Key Laboratory of Intelligent Passenger Service of Civil Aviation-CAAC) Catterall, Benjamin (University of Cambridge) Liu, Chi Harold (Beijing Institute of Technology) Chen, Lydia Y. (TU Delft Data-Intensive Systems) Mortier, Richard (University of Cambridge) Crowcroft, Jon (University of Cambridge) Wang, Liang (University of Cambridge) Date 2022 Abstract With the massive amount of data generated from mobile devices and the increase of computing power of edge devices, the paradigm of Federated Learning has attracted great momentum. In federated learning, distributed and heterogeneous nodes collaborate to learn model parameters. However, while providing benefits such as privacy by design and reduced latency, the heterogeneous network present challenges to the synchronisation methods, or barrier control methods, used in training, regarding system progress and model convergence etc. The design of these barrier mechanisms is critical for the performance and scalability of federated learning systems. We propose a new barrier control technique called Probabilistic Synchronous Parallel (PSP). In contrast to existing mechanisms, it introduces a sampling primitive that composes with existing barrier control mechanisms to produce a family of mechanisms with improved convergence speed and scalability. Our proposal is supported with a convergence analysis of PSP-based SGD algorithm. In practice, we also propose heuristic techniques that further improve the efficiency of PSP. We evaluate the performance of proposed methods using the federated learning specific FEMNSIT dataset. The evaluation results show that PSP can effectively achieve good balance between system efficiency and model accuracy, mitigating the challenge of heterogeneity in federated learning. Subject Federated learningedge computingdistributed computingbarrier control To reference this document use: http://resolver.tudelft.nl/uuid:828dd13b-2d57-4e96-b770-41b4b5c715ea DOI https://doi.org/10.1109/TSC.2021.3109910 Embargo date 2023-07-01 ISSN 1939-1374 Source IEEE Transactions on Services Computing, 15 (2), 614-626 Bibliographical note Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public. Part of collection Institutional Repository Document type journal article Rights © 2022 Jianxin Zhao, Rui Han, Yongkai Yang, Benjamin Catterall, Chi Harold Liu, Lydia Y. Chen, Richard Mortier, Jon Crowcroft, Liang Wang Files PDF Federated_Learning_With_H ... n_Edge.pdf 2.1 MB Close viewer /islandora/object/uuid:828dd13b-2d57-4e96-b770-41b4b5c715ea/datastream/OBJ/view