Federated learning (FL) enables privacy-preserving collaboration among numerous clients for training machine learning models. In FL, a server coordinates model aggregation while preserving data privacy. However, non-identically and independently distributed (non-IID) local data label distributions degrade the performance of the global model. This paper investigates the impact of synthetic data on mitigating non-IID data distributions in federated learning. We explore data-based augmentation techniques, including uniform and minority imputation, utilizing conditional variational autoencoders (CVAEs) to generate synthetic data.
Additionally, we examine a framework-based approach where a pre-trained model, centrally trained on synthetic data, is distributed to clients for finetuning on their original datasets. Our results, which use the binarized MNIST dataset, demonstrate a quality gap between synthetic and original datasets, leading to diminished classification performance when trained on only synthetic data. Integrating both original and synthetic data improves performance on heavily imbalanced label distributions. At the same time, uniform imputation experiments reveal that optimal imputation must strike a balance, with performance degradation being noticeable when datasets consist of more than 45\% synthetic images. Synthetic imputation did not suffer from degradation in the explored range of imputation amounts and achieved an average F1 score improvement of 0.015 over uniform imputation. ... Read more

Federated Learning (FL), is a distributed learning approach where multiple clients collaboratively train a model whilst maintaining data security and privacy. One significant challenge in FL that must be addressed is statistical heterogeneity within the data. This occurs because data across different clients may not come from the same distribution, potentially leading to sub-optimal performance. To address this, we examine how insights gained from a generative model’s latent space can mitigate these problems by adjusting the aggregation weight (influence) assigned to each client during the training process. We leverage information derived from a Variational Autoencoder (VAE) trained in a federated manner and propose a method to modify the aggregation weight of each client in FL. This method considers local discrepancies, resulting from differences between the local latent space distributions and global latent space distributions, together with the dataset sizes of each client. Experiments were conducted on the MNIST and Fashion-MNIST datasets. Our results indicate that our method enhance the model’s performance by up to 6.76% in the best case, in terms of reducing the average test VAE loss and accelerating the convergence of the β-VAE in scenarios characterised by severe data imbalances among clients. It worsens performance when all clients have an equal level of imbalance. The source code for our research is available at https://github.com/FederatedRP2024Delft/
Federated-Learning-PyTorch-Weight-Modification ... Read more

Federated learning stands as an approach to train machine learning models on data residing at multiple clients, but where data must remain private to the client it belongs to. Despite its promise, federated learning faces significant challenges, particularly when dealing with non-IID and non-stationary data. A model trained on non-stationary data can be subject to concept shift, where the data used for training faces a sudden change of concept, leading to a large performance degradation when classifying data under the new concept. This research focuses on comparing the performance of federated and centralized models under such conditions. Our objective is to evaluate the extent to which federated models are more affected by concept shift than their centralized counterparts. Through a series of experiments involving image (CIFAR-10) and tabular data (2-dimensional, linearly separable, binary-classification), we demonstrate that while federated models can achieve performances close to centralized models, they exhibit greater sensitivity to data complexity and distribution shifts. Our findings suggest that, despite centralized models being better than federated ones, the gain in performance from gathering data in one place might not outweigh the privacy concerns. Furthermore, we also find that the accuracy under concept shift is dependent on the performance on original data. ... Read more

Federated Learning (FL) is a machine learning approach that has gained considerable interest over the years. FL allows global models to train without compromising the data privacy of the clients' training datasets by sending the global model to each client to learn the weights and propagating only the learned weights back to a central location. However, it is not without limitations as several challenges hinder the model's performance. One of those challenges is the presence of non-IID (Independent and Identically Distributed) properties in the training data. Most real-world data is non-IID, and this imbalance in data distribution has been shown to significantly affect the model's performance. To address this issue, we propose a generative federated learning by pre-training the global model on synthetic data created by a generative model that follows the collective distribution of all clients' training datasets. Our research shows that this approach bridges the performance gap between IID and non-IID in FL, except for certain extreme non-IID cases. ... Read more

Federated learning (FL) is an innovative approach in machine learning that enables model training across multiple decentralized devices or servers without sharing local data, thus preserving privacy and utilizing decentralized data. However, a significant challenge in FL is handling non-IID (Non-Identical and Independently Distributed) data, which can adversely affect performance. This paper investigates the impact of federated learning on the performance of various generative models, including Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs), specifically in the context of image and tabular data generation tasks. Our study aims to determine how these generative models perform when trained in a federated manner compared to centralized training. We evaluate the models using several metrics, including classifier accuracy on generated images, Earth Mover’s Distance (EMD) for distribution comparison, resemblance, discriminability, downstream utility, and privacy metrics for tabular data. Experiments conducted on the MNIST and CIFAR-10 datasets for image generation, and the Adult and Abalone datasets for tabular data generation, reveal that VAEs exhibit robust and consistent performance across federated and centralized setups. In contrast, GANs show significant performance degradation under federated non-IID conditions. The results indicate that VAEs can effectively address the non-IID data challenge in FL by generating high-quality synthetic data, thereby enhancing model generalizability and stability. The framework used for executing the experiments in this study can be found at https://github.com/alexojica/research-project-experiments. ... Read more

Federated learning enables the construction of machine learning models, while adhering to privacy constraints and without sharing data between different devices. It is achieved by creating a machine learning model on each device that contains data, and then combining these models through an aggregation algorithm without sharing the data. Federated learning is currently a hot topic, and a lot of research has gone into implementing accurate aggregation algorithms. The original algorithm is FedAvg, and since then many different algorithms have been introduced. In this paper, I will compare the performance of five different aggregation algorithms: FedAvg, FedProx, FedYogi, FedMedian and q-FedAvg. The algorithms are compared on different data sets, namely MNIST and a kinase inhibition data set, as well as on different data distributions and number of clients. The experiments indicate that among these five algorithms, FedYogi achieves the best performance, both in terms of highest final accuracy as well as in terms of convergence rate. ... Read more

Federated learning (FL) has emerged as a promis-ing approach for training machine learning models using geographically distributed data. This paper presents a comprehensive comparative study of var-ious machine learning models in the context of FL. The aim is to evaluate the efficacy of these models in different data distribution scenarios and provide
practical insights for practitioners in the field. The findings highlight the performance and limitations of linear and non-linear models on MNIST and Ki-nase datasets. ... Read more

Federated Learning is a machine learning paradigm for decentralized training over different clients. The training happens in rounds where each client learns a specific model which is then aggregated by a central server and passed back to the clients. Since the paradigm’s inception, many frameworks that provide Federated Learning tools and infrastructure have appeared. This leads to the question of ”How do different Federated Learning frameworks compare?”, which is the research question of this paper. The paper’s main contribution will be helping developers new to the Federated Learning field decide between NVidia Flare, OpenFL, and Flower, three popular federated learning frameworks. ... Read more