Decentralized Learning is becoming increasingly popular due to its ability to protect user privacy and scale across large distributed systems. However, when clients leave the network, either by choice or due to failure, their past contributions remain in the model. This raises privacy concerns and may violate the right to be forgotten. In some applications, it is also undesirable to retain the outdated influence of clients that no longer reflect the current state of the system. While Machine Unlearning has seen significant progress in Federated Learning, similar solutions for Decentralized Learning are limited because there is no central server to orchestrate these operations. This work adapts and extends a state-of-the-art Federated Unlearning algorithm, QuickDrop, to operate in a decentralized setting. Our method uses synthetic data to reverse the influence of dropped clients and efficiently restore the model’s generalization performance. It also supports unannounced client crashes and performs reliably in sparse network topologies. We evaluate the algorithm on MNIST and CIFAR-10 using different graph structures and show that it remains competitive with oracle baselines that require access to sensitive data. Finally, we discuss the limitations of our approach and suggest directions for future work in Decentralized Unlearning.

Decentralized Learning (DL) is a key tool for training machine learning models on sensitive, distributed data. However, peer-to-peer model exchange in DL systems exposes participants to privacy attacks. Existing defenses often degrade model utility, introduce communication overhead, or are not applicable to a DL system. Model chunking (splitting a model into parts before sharing them individually) has been proposed as an alternative, but its standalone privacy implications have not been investigated. This work implements and evaluates three distinct model chunking methods (static, cyclic, and random) against two privacy attacks: membership inference and linkability. Our work introduces a Hungarian matching-based enhancement to the linkability attack, and relaxes prior assumptions by evaluating attack success under limited access to neighbor datasets. Our results show that chunking increases vulnerability to membership inference. However, static and random chunking are effective against linkability attacks under specific conditions, particularly when full epochs are used during training.

Decentralized learning (DL) enables a set of nodes to train a model collaboratively without central coordination, offering benefits for privacy and scalability. However, DL struggles to train a high accuracy model when the data distribution is non-independent and identically distributed (non-IID) and when the communication topology is static. To address these issues, we propose DissDL, a fully decentralized topology optimization algorithm for DL where nodes select the peers they exchange models with based on local model dissimilarity. DissDL maintains a fixed in-degree while dynamically adapting the communication graph via gossip-based peer discovery and diversity-driven neighbor selection, improving robustness to data heterogeneity. Experiments on CIFAR-10 and FEMNIST show that DissDL achieves faster convergence, more stable learning, as quantified by lower inter-node variance across nodes, and higher final accuracy than with static topologies and state-of-the-art baselines. For example, on CIFAR-10, DissDL reaches the best accuracy achieved by its most competitive baseline using 1.34 × fewer communication rounds and 1.34 × less communication cost, while maintaining comparable overhead.

Decentralized learning (DL) enables collaborative model training in a distributed fashion without a central server, increasing resilience but still remaining vulnerable to the same adversarial model attacks, that Federated Learning is vulnerable to. In this paper, we test two defense mechanisms, specifically designed to protect fully decentralized learning, in the scenario of chunked models and introduce one new defense. We clearly define the threat model, where malicious peers either try to perform a backdoor attack or an untargeted label flipping. The defenses are Norm Clipping, Sentinel and we propose Adaptive Norm Clipping. We evaluate the effectiveness of these defenses on the CIFAR-10 dataset. Our results indicate that the model attacks (backdoor attack and untargeted label flipping) significantly harm model accuracy on vanilla DL with chunking. While static defenses like Norm Clipping and Adaptive Norm Clipping reduce the impact of the attack, they lower the final test average accuracy in the chunked scenario. Also, robust aggregators like Sentinel fail at mitigating the attack, but do not lower the average test accuracy.

Decentralized learning is a paradigm that enables machine learning in a distributed and decentralized manner. A common challenge in this setting is the presence of non-identically and independently distributed (non-IID) data across clients. Under such conditions, it has been shown that node churn, where clients leave and rejoin the system, leads to reduced generalization performance and slower convergence. This degradation occurs because certain data classes may exist only on a few clients. Thus, if those clients drop out, the global model may lose access to important parts of the data distribution. This setting poses an important question: How can we mitigate the impact of node churn in decentralized learning systems to maintain some persistence of member contributions? To address this challenge, we empirically study the effectiveness of data augmentation, specifically extending local datasets with small synthetic datasets received from neighbors and generated using their local data. We further enhance this approach with supervised contrastive loss applied to synthetic and local data together, to which we refer to as synthetic anchors. Through experiments on the MNIST and CIFAR10 datasets, we demonstrate that data augmentation and synthetic anchors effectively mitigate the effects of churn and help preserve member contribution in decentralized learning.

Federated Learning has gained prominence in recent years, in no small part due to its ability to train Machine Learning models with data from users' devices whilst keeping this data private. Decentralized Federated Learning (DFL) is a branch of Federated Learning (FL) that deals with clients directly communicating with each other as opposed to using a central server. Client mobility describes how users' devices move in the real world, and its effects on the learning performance of Hierarchical Federated Learning (HFL) systems have been found to be significant. However, the effects of client mobility on DFL systems have not been explored. In this work, we fill this research gap. First, we develop a model that can describe client mobility in a DFL system. Then, using synthetic datasets, we show that client mobility has a positive impact on learning performance, which we quantify. Moreover, we show that there is a disparity in learning performance between high-mobility and low-mobility clients when using a baseline model aggregation algorithm. To address this disparity, we propose a new mobility-aware model aggregation algorithm. Our experimental results on synthetic datasets show that our solution reduces the disparity in learning performance between high- and low-mobility clients in the scenarios where this disparity is greatest, with no appreciable downsides in global learning performance.

Federated learning (FL) is a machine learning paradigm where private datasets are distributed among decentralized client devices and model updates are communicated and aggregated to train a shared global model. While providing privacy and scalability benefits, FL systems also face challenges such as client and data heterogeneity, where the training resources and different datasets are non-Independent and Identically Distributed (non-IID). When testing novel FL algorithms or configurations, simulators are used to create controlled environments without needing costly deployments. However, it is important to understand how representative FL simulators are of real-world deployments, and what steps can be taken to bridge the gap between theoretical results and practical implementations. In this paper, we investigate the effects of incorporating traces from a pseudo-real heterogeneous FL deployment in simulated environments. We compare four non-IID attributes, including batch sizes, local epochs, data volume, and data labels, to determine the most influential factors for reproducing the deployment results in simulations. We show that there is an inherent difference between deployments and simulations, despite incorporating identical non-IID conditions. Furthermore, we show that including non-IID data labels in simulations has the most significant impact on recreating the deployment outcome. We also demonstrate that incorporating the other mentioned factors has negligible impact, resulting in similar training performance compared to fully IID simulations. Our results are derived from a 20-client single-server synchronous FL configuration, and additional research is necessary to confirm our findings for larger-scale systems.

Federated Learning is a machine learning paradigm where the computational load for training the model on the server is distributed amongst a pool of clients who only exchange model parameters with the server. Simulation environments try to accurately model all the intricacies of such a system. However, current simulators do not properly impose the concept of simulation time, leading to global model inaccuracies and difficulties of replicating reruns of the simulation, which is most prominent in the asynchronous scenarios. To this purpose, we propose a discrete-event simulator for the central server asynchronous case which timestamps all the events in the system prior to execution, reducing variability in client model updates on the server. We also introduce a log-structure used to keep states of the simulation, making client inspection possible based on time. We evaluate the proposed discrete-event simulator on the baseline simulator of Flower, reducing standard deviation amongst server model updates for 31.5% and improving accuracy with heterogeneous clients in the MNIST case for 3.3% on average.

Federated Learning (FL) systems often suffer from high variability in the final model due to inconsistent training across distributed clients. This paper identifies the problem of high variance in models trained through FL and proposes a novel approach to mitigate this issue through scheduling simulations subject to precedence constraints. By effectively scheduling the execution of client tasks and parameter server updates, we aim to reduce the variance in the final aggregated model. Through a series of experiments, we demonstrate that our proposed scheduling method significantly reduces model variance, while not impacting the time of simulation drastically. Additionally, we propose 2 algorithms to solve the problem of scheduling under precedence constraints - Ant Colony Optimisation, and an Evolutionary Algorithm - to minimize the makespan of simulations.

In federated learning systems, a server maintains a global model trained by a set of clients based on their local datasets. Conventional synchronous FL systems are very sensitive to system heterogeneity since the server needs to wait for the slowest clients in each round. Asynchronous fl partially addresses this bottleneck by dealing with updates once they are received. But with a single server, the system performance would be influenced if the clients are located far from the server and require very high communication costs. Another issue in single-server settings is that the client scale is limited since the server can be overloaded with heavy communication and computation workload. Moreover, a crash on the central server is fatal to the single-server system. Multi-server FLreduces the average communication cost by decreasing the distance between servers and clients. However, the bottleneck brought by the slowest clients still exists in multi-server systems that preserve synchrony, such as Hierarchical FL. The approach we follow in this paper consists in replicating the server in a way that the global training process remains asynchronous. We propose MultiAsync, a novel asynchronous multi-server FL framework that aims to address the single-server and synchronous-system bottleneck.

Federated Learning (FL) is widely favoured in the training of machine learning models due to its privacy-preserving and data diversity benefits. In this research paper, we investigate an extension of FL referred to as Personalized Federated Learning (PFL) for the purpose of training diffusion models. We explore the personalization technique of Trans- fer Learning (TL) and analyse evaluation metrics to capture personalization scores. Transfer Learning has been proven to produce good personalization results under IID and non-IID data distributions. We explore the impact of specific hyperparame- ters and data distribution techniques and examine how the personalization results can be improved even further. We demonstrate that the learning rate and the number of base layers of the convolutional neural network(CNN) form a normal distribution in terms of per-user improvement results. Increasing the number of users creates unstable converges re- sults with the per-user personalization score experi- encing an overall improvement over the pre-trained model independent of the number of users. Our evaluations show that tuning to the optimal hyper- parameter values for specific non-IID data distri- butions produces better personalization scores than other PFL methods.

Federated Continual Learning (FCL) is a emerging field with strong roots in Image classification. However, limited research has been done on its potential in Natural Language Processing and Tabular datasets. With recent developments in A.I. with language models and the widespread use of mobile devices, it becomes relevant to consider FCL’s capabilities in dynamic environments. Our
paper discusses and evaluates the applicability of FCL methods between the domains of Natural Language Processing Tabular Data with Image Processing as a baseline. We use Long-Short Term Memory (LSTM) models, DNN’s and LeNet-5 as models for Sentiment analysis, Tabular classification and Image classification. Through our experiments, we evaluate the average accuracy and backwards transfer of EWC, GEM, their federated variants and the state-of-the-art FCL method FedWEIT. With these methods, sentiment analysis and tabular classification tasks show that image classification reached over 17% higher average accuracy and achieved a 99.5% average increase in knowledge
transfer between tasks. Furthermore, we observe that non-federated continual learning methods on average reach higher accuracies than their federated
counterparts as well as the state-of-the-art methods

The training of diffusion-based models for image generation is predominantly controlled by a select few Big Tech companies, raising concerns about privacy, copyright, and data authority due to the lack of transparency regarding training data. Hence, we propose a federated diffusion model scheme that enables the independent and collaborative training of diffusion models without exposing local data. Our approach adapts the Federated Averaging (FedAvg) algorithm to train a Denoising Diffusion Probabilistic Model (DDPM). Through a novel utilization of the underlying UNet backbone, we achieve a significant reduction of up to 74% in the number of parameters exchanged during training, compared to the naive FedAvg approach, whilst simultaneously maintaining image quality comparable to the centralized setting, as evaluated by the FID score.

Federated learning enables training machine learning models on decentralized data sources without centrally aggregating sensitive information. Continual learning, on the other hand, focuses on learning and adapting to new tasks over time while avoiding the catastrophic forgetting of knowledge from previously encountered tasks. Federated Continual Learning (FCL) addresses this challenge within the framework of federated learning. This thesis investigates how FCL can be made vulnerable to Byzantine attacks (from unpredictable or malicious nodes), which aim to manipulate or corrupt the training process, compromising model performance. We adapt and evaluate four existing attacks from traditional federated learning in the FCL setting. Furthermore, we propose three tailored attacks for FCL are proposed based on the insights gained. Additionally, a novel attack called "Incremental Forgetting" is introduced, which specifically targets the incremental knowledge retention aspect of FCL. Our experimental evaluations of the attacks carried out against various FCL algorithms show that personalizing these towards FCL provides varying degrees of performance benefits, while the novel attack additionally exhibits evidence showing it may be more practical against real-world systems, strengthening its impact on the FCL community. This research contributes to the development of secure and resilient FCL systems to build better defenses against such attacks in the federated learning domain.

In this paper we will consider the Byzantine Reliable Broadcast problem on partially connected net- works. We introduce an routing algorithm for networks with a known topology. It will show that when this is combined with cryptographic signatures, we can use the routing algorithm to create an optimal amount of messages. We will introduce a few algorithms to estimate the routes required for this algorithm. Our simulations show that the amount of messages required for Byzantine Reliable Broadcast can be massively decreased by using this. We will also show that the method of choosing these paths has a huge impact, but come with their own trade- offs.

Increasing digitalisation of society due to technical advancement has increased the appearance and size of cyber- physical systems. These systems require real-time reliable control, which comes with its challenges. These systems need reliable communication despite the presence of attacks or faulty processes and bad connections, which can disrupt the timeliness and correctness of these applications. Furthermore the scale of these systems are becoming increasingly larger as such the cost of connecting all processes that make up these systems become very high. This paper addresses the issues of reliable communication in these conditions while allowing for networks to be partially connected. We show that the protocol discussed can meet real- time requirements of these systems even under conditions of bad connections and low network connectivity.

Distributed systems are networks of nodes depending on each other. However, each network can have multiple faulty nodes, which are either malfunctioning or malicious. Bracha's algorithm allows correct nodes inside the network to agree on certain information, while tolerating a certain amount of faulty nodes. Nodes exchange Send, Echo and Ready message types to inform each other of receiving the message and agreeing on its trustworthiness. The focus of this paper is to present the functional differences of exchanging message paths that messages have crossed instead of the message types. In conclusion, the BMP algorithm showed potential to outperform original Bracha's algorithm in two general cases: networks that have a low probability of successfully transmitting a message, and networks where nodes have a system of trust established which allows them to determine trustworthiness of other nodes. Otherwise, in general usage, original Bracha's algorithms appears to be superior in comparison to the BMP algorithm.

During this research we have replaced Bracha’s layer in the state-of-the-art Bracha-Dolev protocol to improve the performance by decreasing the message complexity of the protocol running on top of a given network topology so long as the requirements stated by Bracha and Dolev are met. Bracha-Dolev is an algorithm that is used to establish a Byzantine fault tolerant communication in a network but it requires a lot of messages to reach consensus. This improvement has been achieved by utilizing a Byzantine Consistent Broadcast algorithm in place of Bracha’s layer: Authenticated Echo Broadcast in order to reduce the message complexity and reduce the network consumption compared to the original optimized Bracha-Dolev algorithm.
Some of the optimizations applied to optimized Bracha-Dolev have also been applied to this new protocol under the hard assumption that the sender is always reliable. As a result, this new protocol is an optimal choice in such instances where these constraints hold and where multiple faulty nodes are present in the system.

Discovering the topology in an unknown network is a fundamental problem for the distributed systems that faces several backlashes due to the proneness of such systems to Byzantine (i.e. arbitrary or malicious) failures. During the past decades, several protocols were developed to allow a network to be resilient up to a certain number of Byzantine processes and guarantee a reliable communication between the correct processes. However, they still suffer from high complexity or unrealistic assumptions which makes them impractical. In this paper modifications made to one of the current state-of-art protocols are presented, modifications that can potentially reduce the load of the network and improve the Reliable Communication layer. We employ empirical analysis in order to prove that reliable communication can still be achieved with these modifications in place and to measure the performance. The findings of this study show that our improved protocol has a reduction of 18.38% in the number of exchanged messages.