Fast Simulation of Federated and Decentralized Learning Algorithms
Scheduling Algorithms for Minimisation of Variability in Federated Learning Simulations
T. Slavov (TU Delft - Electrical Engineering, Mathematics and Computer Science)
B.A. Cox – Mentor (TU Delft - Data-Intensive Systems)
Jérémie Decouchant – Mentor (TU Delft - Data-Intensive Systems)
Qing Wang – Graduation committee member (TU Delft - Embedded Systems)
More Info
expand_more
Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.
Abstract
Federated Learning (FL) systems often suffer from high variability in the final model due to inconsistent training across distributed clients. This paper identifies the problem of high variance in models trained through FL and proposes a novel approach to mitigate this issue through scheduling simulations subject to precedence constraints. By effectively scheduling the execution of client tasks and parameter server updates, we aim to reduce the variance in the final aggregated model. Through a series of experiments, we demonstrate that our proposed scheduling method significantly reduces model variance, while not impacting the time of simulation drastically. Additionally, we propose 2 algorithms to solve the problem of scheduling under precedence constraints - Ant Colony Optimisation, and an Evolutionary Algorithm - to minimize the makespan of simulations.