MAPL: Model Agnostic Peer-to-Peer Learning

Master thesis (2023)

Authors

S. Mukherjee Electrical Engineering, Mathematics and Computer Science

Contributors

H. Jamali-Rad Pattern Recognition and Bioinformatics - EEMCS (supervisor 1)
J.C. van Gemert Pattern Recognition and Bioinformatics - EEMCS (supervisor 2)
G.J.T. Leus Signal Processing Systems - EEMCS (supervisor 2)
Andrea Simonetto ENSTA Paris Institut Polytechnique de Paris (supervisor 1)
Faculty
Electrical Engineering, Mathematics and Computer Science
Copyright: © 2023 Sayak Mukherjee
Deep Learning Federated Learning Decentralized learning
More Info
expand_more

Published Date

31-08-2023

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Abstract

Current methods in Federated and Decentralized learning presume that all clients share the same model architecture, assuming model homogeneity. However, in practice, this assumption may not always hold due to hardware differences. While prior research has addressed model heterogeneity in Federated Learning, it remains unexplored in fully decentralized or peer-to-peer settings. Therefore, in this paper, we investigate a real-world yet challenging situation involving model heterogeneity in a fully decentralized context. Furthermore, we introduced a Model Agnostic Peer-to-peer Learning (MAPL) framework, which allows simultaneous learning of heterogeneous personalized models. Additionally, we define a graph learning objective to infer optimal collaboration weights based on task similarity. Experiments reveal that even in this challenging scenario, MAPL delivers competitive results while being communication efficient owing to the sparse collaboration graph in both model homogeneous and heterogeneous settings.