Federated Learning With Taskonomy for Non-IID Data

Journal Article (2022)
Author(s)

H Jamali-Rad (Shell Global Solutions International B.V., TU Delft - Pattern Recognition and Bioinformatics)

Mohammad Abdizadeh (Myant Inc.)

Anuj Sing (Student TU Delft)

Research Group
Pattern Recognition and Bioinformatics
Copyright
© 2022 H. Jamali-Rad, Mohammad Abdizadeh, Anuj Sing
DOI related publication
https://doi.org/10.1109/TNNLS.2022.3152581
More Info
expand_more
Publication Year
2022
Language
English
Copyright
© 2022 H. Jamali-Rad, Mohammad Abdizadeh, Anuj Sing
Research Group
Pattern Recognition and Bioinformatics
Issue number
11
Volume number
34
Pages (from-to)
8719-8730
Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Abstract

Classical federated learning approaches incur significant performance degradation in the presence of non-independent and identically distributed (non-IID) client data. A possible direction to address this issue is forming clusters of clients with roughly IID data. Most solutions following this direction are iterative and relatively slow, also prone to convergence issues in discovering underlying cluster formations. We introduce federated learning with taskonomy (FLT) that generalizes this direction by learning the task relatedness between clients for more efficient federated aggregation of heterogeneous data. In a one-off process, the server provides the clients with a pretrained (and fine-tunable) encoder to compress their data into a latent representation and transmit the signature of their data back to the server. The server then learns the task relatedness among clients via manifold learning and performs a generalization of federated averaging. FLT can flexibly handle a generic client relatedness graph, when there are no explicit clusters of clients, as well as efficiently decompose it into (disjoint) clusters for clustered federated learning. We demonstrate that FLT not only outperforms the existing state-of-the-art baselines in non-IID scenarios but also offers improved fairness across clients. Our codebase can be found at: https://github.com/hjraad/FLT/