YC
Y. Chen
info
Please Note
<p>This page displays the records of the person named above and is not linked to a unique person identifier. This record may need to be merged to a profile.</p>
67 records found
1
Master thesis
(2026)
-
F. Nardi Dei da Filicaia Dotti, Y. Chen, R.R. Venkatesha Prasad, Basile Lewandowsky
Text-to-image diffusion models have advanced significantly in recent years. Different models show strong performance across various generation tasks. Choosing the right model is becoming increasingly important since no single model consistently outperforms others in all cases. However, existing model selection approaches are typically evaluated only at the dataset level. Such evaluation overlooks prompt-level variation, where different models may excel on different prompts. In this thesis, we investigate diffusion model selection during inference. The goal is to pick the best model for each individual prompt. We first examine the online setting, where model selection occurs adaptively during deployment. In this context, we create a framework for online diffusion model selection and test it against recent methods from the literature. Our findings show that this approach outperforms existing online selection strategies, highlighting the benefits of prompt-aware model selection. In addition to the online setting, we present an offline approach to diffusion model selection, where decisions are made without online interaction. Overall, this thesis claims that diffusion model selection should be viewed as a prompt-level decision rather than a dataset-level comparison. By exploring both online and offline settings and providing empirical results alongside detailed ablations, we aim to promote a more practical and adaptable approach to diffusion model selection.
...
Text-to-image diffusion models have advanced significantly in recent years. Different models show strong performance across various generation tasks. Choosing the right model is becoming increasingly important since no single model consistently outperforms others in all cases. However, existing model selection approaches are typically evaluated only at the dataset level. Such evaluation overlooks prompt-level variation, where different models may excel on different prompts. In this thesis, we investigate diffusion model selection during inference. The goal is to pick the best model for each individual prompt. We first examine the online setting, where model selection occurs adaptively during deployment. In this context, we create a framework for online diffusion model selection and test it against recent methods from the literature. Our findings show that this approach outperforms existing online selection strategies, highlighting the benefits of prompt-aware model selection. In addition to the online setting, we present an offline approach to diffusion model selection, where decisions are made without online interaction. Overall, this thesis claims that diffusion model selection should be viewed as a prompt-level decision rather than a dataset-level comparison. By exploring both online and offline settings and providing empirical results alongside detailed ablations, we aim to promote a more practical and adaptable approach to diffusion model selection.
Federated learning (FL) has emerged as an important paradigm in distributed machine learning, enabling collaborative model training across decentralized devices while preserving data privacy. FL’s privacy-preserving nature – where raw data remains on local devices and only model updates are shared – has made it suitable in sensitive domains like healthcare and finance. However, the decentralized framework introduces fundamental challenges that threaten its reliability and adoption. Data heterogeneity, security threats, and privacy leakage risks create critical vulnerabilities that demand robust solutions.
To study such vulnerabilities, this thesis considers two kinds of parties: the clients and the servers. Clients act as data owners that perform localized computations and share only model parameters, thereby preserving raw data privacy, yet they introduce vulnerabilities through potential malicious behaviors (e.g., data/model poisoning attacks) or unreliable contributions due to data quality. In contrast, the server, while facilitating model convergence through aggregation, poses inherent privacy risks by potentially inferring sensitive client information from shared gradients, even without direct data access. These two parties create a dual-threat landscape: clients may compromise model performance through adversarial manipulations, while servers break confidentiality via reconstruction methods.... ...
To study such vulnerabilities, this thesis considers two kinds of parties: the clients and the servers. Clients act as data owners that perform localized computations and share only model parameters, thereby preserving raw data privacy, yet they introduce vulnerabilities through potential malicious behaviors (e.g., data/model poisoning attacks) or unreliable contributions due to data quality. In contrast, the server, while facilitating model convergence through aggregation, poses inherent privacy risks by potentially inferring sensitive client information from shared gradients, even without direct data access. These two parties create a dual-threat landscape: clients may compromise model performance through adversarial manipulations, while servers break confidentiality via reconstruction methods.... ...
Federated learning (FL) has emerged as an important paradigm in distributed machine learning, enabling collaborative model training across decentralized devices while preserving data privacy. FL’s privacy-preserving nature – where raw data remains on local devices and only model updates are shared – has made it suitable in sensitive domains like healthcare and finance. However, the decentralized framework introduces fundamental challenges that threaten its reliability and adoption. Data heterogeneity, security threats, and privacy leakage risks create critical vulnerabilities that demand robust solutions.
To study such vulnerabilities, this thesis considers two kinds of parties: the clients and the servers. Clients act as data owners that perform localized computations and share only model parameters, thereby preserving raw data privacy, yet they introduce vulnerabilities through potential malicious behaviors (e.g., data/model poisoning attacks) or unreliable contributions due to data quality. In contrast, the server, while facilitating model convergence through aggregation, poses inherent privacy risks by potentially inferring sensitive client information from shared gradients, even without direct data access. These two parties create a dual-threat landscape: clients may compromise model performance through adversarial manipulations, while servers break confidentiality via reconstruction methods....
To study such vulnerabilities, this thesis considers two kinds of parties: the clients and the servers. Clients act as data owners that perform localized computations and share only model parameters, thereby preserving raw data privacy, yet they introduce vulnerabilities through potential malicious behaviors (e.g., data/model poisoning attacks) or unreliable contributions due to data quality. In contrast, the server, while facilitating model convergence through aggregation, poses inherent privacy risks by potentially inferring sensitive client information from shared gradients, even without direct data access. These two parties create a dual-threat landscape: clients may compromise model performance through adversarial manipulations, while servers break confidentiality via reconstruction methods....
Knowledge distillation, the process of transferring learned knowledge from one target (data or model) to a substitute, has become essential for improving efficiency to reduce computational cost while maintaining accuracy. However, knowledge differs across data quality (noisy/clean), task types (classification/generation), and model accessibility (black-box/white-box). These variations introduce distinct challenges. Thus, this thesis systematically investigates how to distill knowledge from multiple sources—noisy crowdsourced labels, black-box classifiers, white-box generative models, and more complex diffusion models—to improve both robustness and efficiency.
To address these challenges, this thesis proposes five research questions, combining theoretical analysis with empirical validation across diverse machine learning scenarios. The first challenge considers noisy crowdsourced labels, where non-professional workers introduce errors that degrade model performance. It calls for online aggregation methods to process data incrementally rather than in one go on a whole set. The second vulnerability involves black-box model distillation without real data, where efficiently generating high-quality synthetic queries remains difficult. The third challenge extends this to incorporating semantic information from public data, aiming to reduce the number of queries typically required for effective distillation. The fourth investigates generative model distillation, asking whether dark knowledge (inference probabilities) exists beyond final outputs and how it improves generalization. The fifth examines diffusion models, whose multi-step Markov chain structure introduces unique difficulties for distillation and sampling acceleration.
Chapter 2 tackles distilling knowledge from noisy crowdsourced labels. Unlike offline aggregation methods requiring all labels at once, we propose BILA , an online framework that processes label chunks incrementally using a confusion matrix-based neural network model which can be trained by first-order stochastic optimizers. BILA achieves higher accuracy than existing offline algorithms, enabling robust real-time label cleaning.
Chapter 3 addresses black-box distillation without access to real training data. Existing methods only explore the input space inefficiently. We propose TANDEMGAN, which combines exploration, which generates diverse synthetic queries, with exploitation, which focuses on high-confidence queries. This tandem architecture enables effective substitute model training in general adversarial scenarios where only class labels are available.
Chapter 4 further improves black-box efficiency by incorporating semantic information from public data knowledge. We introduce AEDM, which leverages pre-trained diffusion models to generate semantically rich query images resembling real data. By optimizing the input noise of the diffusion model based on substitute model feedback, AEDM achieves superior distillation accuracy with significantly fewer queries and extends to federated learning settings.
...
To address these challenges, this thesis proposes five research questions, combining theoretical analysis with empirical validation across diverse machine learning scenarios. The first challenge considers noisy crowdsourced labels, where non-professional workers introduce errors that degrade model performance. It calls for online aggregation methods to process data incrementally rather than in one go on a whole set. The second vulnerability involves black-box model distillation without real data, where efficiently generating high-quality synthetic queries remains difficult. The third challenge extends this to incorporating semantic information from public data, aiming to reduce the number of queries typically required for effective distillation. The fourth investigates generative model distillation, asking whether dark knowledge (inference probabilities) exists beyond final outputs and how it improves generalization. The fifth examines diffusion models, whose multi-step Markov chain structure introduces unique difficulties for distillation and sampling acceleration.
Chapter 2 tackles distilling knowledge from noisy crowdsourced labels. Unlike offline aggregation methods requiring all labels at once, we propose BILA , an online framework that processes label chunks incrementally using a confusion matrix-based neural network model which can be trained by first-order stochastic optimizers. BILA achieves higher accuracy than existing offline algorithms, enabling robust real-time label cleaning.
Chapter 3 addresses black-box distillation without access to real training data. Existing methods only explore the input space inefficiently. We propose TANDEMGAN, which combines exploration, which generates diverse synthetic queries, with exploitation, which focuses on high-confidence queries. This tandem architecture enables effective substitute model training in general adversarial scenarios where only class labels are available.
Chapter 4 further improves black-box efficiency by incorporating semantic information from public data knowledge. We introduce AEDM, which leverages pre-trained diffusion models to generate semantically rich query images resembling real data. By optimizing the input noise of the diffusion model based on substitute model feedback, AEDM achieves superior distillation accuracy with significantly fewer queries and extends to federated learning settings.
Chapter 5 provides a theoretical analysis for generative model distillation. We derive a risk bound demonstrating that incorporating dark knowledge, which is the underlying conditional distributions between inputs and outputs, improves generalization. Our DKtill framework aligns student and teacher probabilistic relationships, outperforming methods that rely solely on final outputs across GANs and VAEs.
Chapter 6 targets diffusion models. Unlike prior work that merely mimics outputs,
we propose SFDDM, which aligns the Markov chains of student and teacher models. By reparameterizing intermediate inputs and minimizing differences in both output and hidden variables, SFDDM produces high-quality samples with significantly fewer steps. SFDDM enables the distillation from the teacher to a student model with any desired step size.
Finally, we summarize the conclusions of this thesis. Two overarching findings emerge:
(1) robust distillation requires identifying and extracting the most valuable information from each source, whether through elaborated inputs, probabilistic relationships, or structural alignment; (2) efficient distillation demands methods that match the constraints of each setting, including incremental processing for noisy data, semantic priors for black-box queries, and chain alignment for diffusion models. We also discuss limitations, including narrow architectural choices, dependency on specific probability approximations, and computational overhead, while outlining future directions such as exploring more powerful networks, alternative quality criteria, rigorous proofs for non-probabilistic generators, and extension to guided or latent diffusion models.
Knowledge distillation, the process of transferring learned knowledge from one target (data or model) to a substitute, has become essential for improving efficiency to reduce computational cost while maintaining accuracy. However, knowledge differs across data quality (noisy/clean), task types (classification/generation), and model accessibility (black-box/white-box). These variations introduce distinct challenges. Thus, this thesis systematically investigates how to distill knowledge from multiple sources—noisy crowdsourced labels, black-box classifiers, white-box generative models, and more complex diffusion models—to improve both robustness and efficiency.
To address these challenges, this thesis proposes five research questions, combining theoretical analysis with empirical validation across diverse machine learning scenarios. The first challenge considers noisy crowdsourced labels, where non-professional workers introduce errors that degrade model performance. It calls for online aggregation methods to process data incrementally rather than in one go on a whole set. The second vulnerability involves black-box model distillation without real data, where efficiently generating high-quality synthetic queries remains difficult. The third challenge extends this to incorporating semantic information from public data, aiming to reduce the number of queries typically required for effective distillation. The fourth investigates generative model distillation, asking whether dark knowledge (inference probabilities) exists beyond final outputs and how it improves generalization. The fifth examines diffusion models, whose multi-step Markov chain structure introduces unique difficulties for distillation and sampling acceleration.
Chapter 2 tackles distilling knowledge from noisy crowdsourced labels. Unlike offline aggregation methods requiring all labels at once, we propose BILA , an online framework that processes label chunks incrementally using a confusion matrix-based neural network model which can be trained by first-order stochastic optimizers. BILA achieves higher accuracy than existing offline algorithms, enabling robust real-time label cleaning.
Chapter 3 addresses black-box distillation without access to real training data. Existing methods only explore the input space inefficiently. We propose TANDEMGAN, which combines exploration, which generates diverse synthetic queries, with exploitation, which focuses on high-confidence queries. This tandem architecture enables effective substitute model training in general adversarial scenarios where only class labels are available.
Chapter 4 further improves black-box efficiency by incorporating semantic information from public data knowledge. We introduce AEDM, which leverages pre-trained diffusion models to generate semantically rich query images resembling real data. By optimizing the input noise of the diffusion model based on substitute model feedback, AEDM achieves superior distillation accuracy with significantly fewer queries and extends to federated learning settings.
To address these challenges, this thesis proposes five research questions, combining theoretical analysis with empirical validation across diverse machine learning scenarios. The first challenge considers noisy crowdsourced labels, where non-professional workers introduce errors that degrade model performance. It calls for online aggregation methods to process data incrementally rather than in one go on a whole set. The second vulnerability involves black-box model distillation without real data, where efficiently generating high-quality synthetic queries remains difficult. The third challenge extends this to incorporating semantic information from public data, aiming to reduce the number of queries typically required for effective distillation. The fourth investigates generative model distillation, asking whether dark knowledge (inference probabilities) exists beyond final outputs and how it improves generalization. The fifth examines diffusion models, whose multi-step Markov chain structure introduces unique difficulties for distillation and sampling acceleration.
Chapter 2 tackles distilling knowledge from noisy crowdsourced labels. Unlike offline aggregation methods requiring all labels at once, we propose BILA , an online framework that processes label chunks incrementally using a confusion matrix-based neural network model which can be trained by first-order stochastic optimizers. BILA achieves higher accuracy than existing offline algorithms, enabling robust real-time label cleaning.
Chapter 3 addresses black-box distillation without access to real training data. Existing methods only explore the input space inefficiently. We propose TANDEMGAN, which combines exploration, which generates diverse synthetic queries, with exploitation, which focuses on high-confidence queries. This tandem architecture enables effective substitute model training in general adversarial scenarios where only class labels are available.
Chapter 4 further improves black-box efficiency by incorporating semantic information from public data knowledge. We introduce AEDM, which leverages pre-trained diffusion models to generate semantically rich query images resembling real data. By optimizing the input noise of the diffusion model based on substitute model feedback, AEDM achieves superior distillation accuracy with significantly fewer queries and extends to federated learning settings.
Chapter 5 provides a theoretical analysis for generative model distillation. We derive a risk bound demonstrating that incorporating dark knowledge, which is the underlying conditional distributions between inputs and outputs, improves generalization. Our DKtill framework aligns student and teacher probabilistic relationships, outperforming methods that rely solely on final outputs across GANs and VAEs.
Chapter 6 targets diffusion models. Unlike prior work that merely mimics outputs,
we propose SFDDM, which aligns the Markov chains of student and teacher models. By reparameterizing intermediate inputs and minimizing differences in both output and hidden variables, SFDDM produces high-quality samples with significantly fewer steps. SFDDM enables the distillation from the teacher to a student model with any desired step size.
Finally, we summarize the conclusions of this thesis. Two overarching findings emerge:
(1) robust distillation requires identifying and extracting the most valuable information from each source, whether through elaborated inputs, probabilistic relationships, or structural alignment; (2) efficient distillation demands methods that match the constraints of each setting, including incremental processing for noisy data, semantic priors for black-box queries, and chain alignment for diffusion models. We also discuss limitations, including narrow architectural choices, dependency on specific probability approximations, and computational overhead, while outlining future directions such as exploring more powerful networks, alternative quality criteria, rigorous proofs for non-probabilistic generators, and extension to guided or latent diffusion models.
Learn together over time
Distributed Multi-frequency time series framework
Modern industrial systems, from wind-farm monitoring to economic indicators like GDP generate vast amounts of time series data from diverse sources. These data streams are sampled at varying and often inconsistent frequencies, presenting challenges for accurate forecasting. Furthermore, in many real-world scenarios, data are distributed across nodes or tasks, introducing complications due to heterogeneity across tasks. Existing forecasting approaches typically address frequency misalignment and decentralized learning as separate problems, limiting their ability to model real-world deployments effectively. We propose CrossFreqNet, a unified multitask encoder–decoder architecture that addresses both challenges: (i.) integrating multi-frequency data streams without the need of up or down sampling to match frequency, preserving signal integrity and (ii.) introducing GradBal, a gradient-balancing mechanism that mitigates learning conflicts caused by task heterogeneity and promoting stable convergence across tasks in a distributed learning environment. Across four public benchmarks and one industrial dataset, our model reduces forecasting errors by up to 72% over the best multi-task baseline (UniTS) and up to 48% over PCGrad, a SOTA gradient conflict mitigation method. Code is made available at https://github.com/arc-arnob/TS-MTL/.
...
Modern industrial systems, from wind-farm monitoring to economic indicators like GDP generate vast amounts of time series data from diverse sources. These data streams are sampled at varying and often inconsistent frequencies, presenting challenges for accurate forecasting. Furthermore, in many real-world scenarios, data are distributed across nodes or tasks, introducing complications due to heterogeneity across tasks. Existing forecasting approaches typically address frequency misalignment and decentralized learning as separate problems, limiting their ability to model real-world deployments effectively. We propose CrossFreqNet, a unified multitask encoder–decoder architecture that addresses both challenges: (i.) integrating multi-frequency data streams without the need of up or down sampling to match frequency, preserving signal integrity and (ii.) introducing GradBal, a gradient-balancing mechanism that mitigates learning conflicts caused by task heterogeneity and promoting stable convergence across tasks in a distributed learning environment. Across four public benchmarks and one industrial dataset, our model reduces forecasting errors by up to 72% over the best multi-task baseline (UniTS) and up to 48% over PCGrad, a SOTA gradient conflict mitigation method. Code is made available at https://github.com/arc-arnob/TS-MTL/.
Transformer-Based Synthetic Relational Data
Closing the Gap Between Diffusion-Based and Transformer-Based Synthetic Relational Data Generation
Data sharing for research and industrial applications faces significant challenges due to privacy constraints and regulatory requirements, driving the need for high-quality synthetic alternatives.
Recent advances in synthetic data generation have demonstrated considerable success for single-table datasets, with emerging research extending these capabilities to multi-table relational scenarios.
While transformer and diffusion architectures achieve state-of-the-art performance in single-table generation, a notable performance gap emerges when applied to relational data, where diffusion approaches consistently outperform transformer-based methods.
This thesis examines the factors contributing to this performance difference, conducting an evaluation using multiple baselines across both single and relational tabular datasets, with REaLTabformer and ClavaDDPM as state-of-the-art transformer- and diffusion-based approaches, respectively.
Our investigation reveals that the performance can mainly be attributed to the inadequate processing of contextual relationships and suboptimal strategies for representing inter-table dependencies in transformer-based models.
To close this gap, we introduce two changes for transformer-based models: layer sharing to enhance parameter utilization and contextual encoding to better preserve the relational structure.
These changes provide insight into the key design principles behind effective synthetic relational data generation using transformer-based models, particularly the need for architectures that account for context and facilitate practical knowledge transfer.
The proposed methods result in substantial performance improvements, with a 1.52-fold improvement in Logistic Detection and a 1.94-fold reduction in the Discriminator Measure metric.
...
Recent advances in synthetic data generation have demonstrated considerable success for single-table datasets, with emerging research extending these capabilities to multi-table relational scenarios.
While transformer and diffusion architectures achieve state-of-the-art performance in single-table generation, a notable performance gap emerges when applied to relational data, where diffusion approaches consistently outperform transformer-based methods.
This thesis examines the factors contributing to this performance difference, conducting an evaluation using multiple baselines across both single and relational tabular datasets, with REaLTabformer and ClavaDDPM as state-of-the-art transformer- and diffusion-based approaches, respectively.
Our investigation reveals that the performance can mainly be attributed to the inadequate processing of contextual relationships and suboptimal strategies for representing inter-table dependencies in transformer-based models.
To close this gap, we introduce two changes for transformer-based models: layer sharing to enhance parameter utilization and contextual encoding to better preserve the relational structure.
These changes provide insight into the key design principles behind effective synthetic relational data generation using transformer-based models, particularly the need for architectures that account for context and facilitate practical knowledge transfer.
The proposed methods result in substantial performance improvements, with a 1.52-fold improvement in Logistic Detection and a 1.94-fold reduction in the Discriminator Measure metric.
...
Data sharing for research and industrial applications faces significant challenges due to privacy constraints and regulatory requirements, driving the need for high-quality synthetic alternatives.
Recent advances in synthetic data generation have demonstrated considerable success for single-table datasets, with emerging research extending these capabilities to multi-table relational scenarios.
While transformer and diffusion architectures achieve state-of-the-art performance in single-table generation, a notable performance gap emerges when applied to relational data, where diffusion approaches consistently outperform transformer-based methods.
This thesis examines the factors contributing to this performance difference, conducting an evaluation using multiple baselines across both single and relational tabular datasets, with REaLTabformer and ClavaDDPM as state-of-the-art transformer- and diffusion-based approaches, respectively.
Our investigation reveals that the performance can mainly be attributed to the inadequate processing of contextual relationships and suboptimal strategies for representing inter-table dependencies in transformer-based models.
To close this gap, we introduce two changes for transformer-based models: layer sharing to enhance parameter utilization and contextual encoding to better preserve the relational structure.
These changes provide insight into the key design principles behind effective synthetic relational data generation using transformer-based models, particularly the need for architectures that account for context and facilitate practical knowledge transfer.
The proposed methods result in substantial performance improvements, with a 1.52-fold improvement in Logistic Detection and a 1.94-fold reduction in the Discriminator Measure metric.
Recent advances in synthetic data generation have demonstrated considerable success for single-table datasets, with emerging research extending these capabilities to multi-table relational scenarios.
While transformer and diffusion architectures achieve state-of-the-art performance in single-table generation, a notable performance gap emerges when applied to relational data, where diffusion approaches consistently outperform transformer-based methods.
This thesis examines the factors contributing to this performance difference, conducting an evaluation using multiple baselines across both single and relational tabular datasets, with REaLTabformer and ClavaDDPM as state-of-the-art transformer- and diffusion-based approaches, respectively.
Our investigation reveals that the performance can mainly be attributed to the inadequate processing of contextual relationships and suboptimal strategies for representing inter-table dependencies in transformer-based models.
To close this gap, we introduce two changes for transformer-based models: layer sharing to enhance parameter utilization and contextual encoding to better preserve the relational structure.
These changes provide insight into the key design principles behind effective synthetic relational data generation using transformer-based models, particularly the need for architectures that account for context and facilitate practical knowledge transfer.
The proposed methods result in substantial performance improvements, with a 1.52-fold improvement in Logistic Detection and a 1.94-fold reduction in the Discriminator Measure metric.
This survey explores state-of-the-art advancements in accelerating diffusion models, focusing on techniques to address their computational and memory inefficiencies. Diffusion models have achieved remarkable success in generative AI, surpassing prior paradigms like GANs in various applications, including image synthesis, text-to-image generation, video generation and more. However, their reliance on a large number of sequential sampling steps significantly hinders their efficiency compared to other generative approaches. This survey categorises and analyses 11 recent works aimed at overcoming these challenges, including quantization techniques, knowledge distillation, and distributed parallel sampling. Through this survey, we aim to provide an understanding, intuition, theory and tradeoffs behind these techniques. Finally, this work offers a valuable reference for researchers and professionals seeking to enhance or utilise fast diffusion model architectures, providing a clear overview of benchmarking parameters used for each of these works.
...
This survey explores state-of-the-art advancements in accelerating diffusion models, focusing on techniques to address their computational and memory inefficiencies. Diffusion models have achieved remarkable success in generative AI, surpassing prior paradigms like GANs in various applications, including image synthesis, text-to-image generation, video generation and more. However, their reliance on a large number of sequential sampling steps significantly hinders their efficiency compared to other generative approaches. This survey categorises and analyses 11 recent works aimed at overcoming these challenges, including quantization techniques, knowledge distillation, and distributed parallel sampling. Through this survey, we aim to provide an understanding, intuition, theory and tradeoffs behind these techniques. Finally, this work offers a valuable reference for researchers and professionals seeking to enhance or utilise fast diffusion model architectures, providing a clear overview of benchmarking parameters used for each of these works.
Many fields rely on scarce and sensitive time series data, such as patient health records. Privacy regulations often make sharing such data challenging, slowing research progress. Synthetic time series offer a potential solution by replicating statistical characteristics of real data without revealing private information. Yet, they introduce new risks, as synthetic data may be mistaken for real. Watermarking can mitigate this by embedding a machine-detectable signal that preserves data quality. For such methods to be effective, watermarks must be robust to removal attempts. Existing research lacks direct comparisons of generative models for time series synthesis and watermarking. Furthermore, they only evaluate watermark robustness against time-domain attacks. Attacks in other domains, such as the frequency domain, remain unexplored. In order to address these gaps, this thesis investigates three key questions. First, which generative models are best suited for time series synthesis. Second, whether latent diffusion models (LDMs) can support watermarking. Lastly, how robust existing diffusion watermarks are against adversarial attacks.
A comparative study between GPT-based models and diffusion models showed that diffusion models produce synthetic data of higher quality. LDMs were then evaluated as a potential alternative. Their reliance on a variational autoencoder led to low quality outputs. Hence, standard diffusion models were elected as the superior watermarking candidate. Finally, we introduced an extended set of time-, frequency-, and time-frequency domain attacks to asses watermark robustness. TimeWak emerged as the most robust watermark. However, our extended attack suite revealed new vulnerabilities in all watermarks, highlighting the importance of comprehensive robustness evaluations. ...
A comparative study between GPT-based models and diffusion models showed that diffusion models produce synthetic data of higher quality. LDMs were then evaluated as a potential alternative. Their reliance on a variational autoencoder led to low quality outputs. Hence, standard diffusion models were elected as the superior watermarking candidate. Finally, we introduced an extended set of time-, frequency-, and time-frequency domain attacks to asses watermark robustness. TimeWak emerged as the most robust watermark. However, our extended attack suite revealed new vulnerabilities in all watermarks, highlighting the importance of comprehensive robustness evaluations. ...
Many fields rely on scarce and sensitive time series data, such as patient health records. Privacy regulations often make sharing such data challenging, slowing research progress. Synthetic time series offer a potential solution by replicating statistical characteristics of real data without revealing private information. Yet, they introduce new risks, as synthetic data may be mistaken for real. Watermarking can mitigate this by embedding a machine-detectable signal that preserves data quality. For such methods to be effective, watermarks must be robust to removal attempts. Existing research lacks direct comparisons of generative models for time series synthesis and watermarking. Furthermore, they only evaluate watermark robustness against time-domain attacks. Attacks in other domains, such as the frequency domain, remain unexplored. In order to address these gaps, this thesis investigates three key questions. First, which generative models are best suited for time series synthesis. Second, whether latent diffusion models (LDMs) can support watermarking. Lastly, how robust existing diffusion watermarks are against adversarial attacks.
A comparative study between GPT-based models and diffusion models showed that diffusion models produce synthetic data of higher quality. LDMs were then evaluated as a potential alternative. Their reliance on a variational autoencoder led to low quality outputs. Hence, standard diffusion models were elected as the superior watermarking candidate. Finally, we introduced an extended set of time-, frequency-, and time-frequency domain attacks to asses watermark robustness. TimeWak emerged as the most robust watermark. However, our extended attack suite revealed new vulnerabilities in all watermarks, highlighting the importance of comprehensive robustness evaluations.
A comparative study between GPT-based models and diffusion models showed that diffusion models produce synthetic data of higher quality. LDMs were then evaluated as a potential alternative. Their reliance on a variational autoencoder led to low quality outputs. Hence, standard diffusion models were elected as the superior watermarking candidate. Finally, we introduced an extended set of time-, frequency-, and time-frequency domain attacks to asses watermark robustness. TimeWak emerged as the most robust watermark. However, our extended attack suite revealed new vulnerabilities in all watermarks, highlighting the importance of comprehensive robustness evaluations.
Bayesian Networks (BNs) are widely utilized across various industrial sectors to optimize processes, with an emerging focus on the collaboration across multiple parties. While most realistic scenarios require handling a mixture of categorical and continuous data simultaneously, the current state-of-the-art only supports collaborative inference on purely discrete models. The Junction Tree enables efficient and accurate inference on hybrid models but has not been implemented for confidential scenarios yet. To address this gap, we introduce Hybrid CCJT, an innovative framework for confidential multiparty inference in hybrid domains, offering: (i) a method to construct a collaborative, strongly-rooted junction tree for efficient and secure inference, (ii) a confidential-
preserving inference protocol for Hybrid BNs, (iii) an optimized message-passing scheme that
improves communication efficiency even in the purely discrete domain. Our extensive evaluation
show that Hybrid CCJT improves the predictive accuracy of continuous target variables by an average of 32% in Mean Squared Error and reduce the communication cost up to 86-fold, against the best state-of-the-art baseline. ...
preserving inference protocol for Hybrid BNs, (iii) an optimized message-passing scheme that
improves communication efficiency even in the purely discrete domain. Our extensive evaluation
show that Hybrid CCJT improves the predictive accuracy of continuous target variables by an average of 32% in Mean Squared Error and reduce the communication cost up to 86-fold, against the best state-of-the-art baseline. ...
Bayesian Networks (BNs) are widely utilized across various industrial sectors to optimize processes, with an emerging focus on the collaboration across multiple parties. While most realistic scenarios require handling a mixture of categorical and continuous data simultaneously, the current state-of-the-art only supports collaborative inference on purely discrete models. The Junction Tree enables efficient and accurate inference on hybrid models but has not been implemented for confidential scenarios yet. To address this gap, we introduce Hybrid CCJT, an innovative framework for confidential multiparty inference in hybrid domains, offering: (i) a method to construct a collaborative, strongly-rooted junction tree for efficient and secure inference, (ii) a confidential-
preserving inference protocol for Hybrid BNs, (iii) an optimized message-passing scheme that
improves communication efficiency even in the purely discrete domain. Our extensive evaluation
show that Hybrid CCJT improves the predictive accuracy of continuous target variables by an average of 32% in Mean Squared Error and reduce the communication cost up to 86-fold, against the best state-of-the-art baseline.
preserving inference protocol for Hybrid BNs, (iii) an optimized message-passing scheme that
improves communication efficiency even in the purely discrete domain. Our extensive evaluation
show that Hybrid CCJT improves the predictive accuracy of continuous target variables by an average of 32% in Mean Squared Error and reduce the communication cost up to 86-fold, against the best state-of-the-art baseline.
Attacking Federated Time Series Forecasting Models
Reconstructing Private Household Energy Data during Federated Learning with Gradient Inversion Attacks
Federated learning for time series forecasting enables clients with privacy-sensitive time series data to collaboratively learn accurate forecasting models, e.g., in energy load prediction.
Unfortunately, privacy risks in federated learning persist, as servers can potentially reconstruct clients' training data through gradient inversion attacks.
While gradient inversion attacks are demonstrated for image, text and tabular classification tasks, little is known for time series regression tasks.
In this paper, we first conduct an extensive empirical study on inverting time series data across 4 time series forecasting models and 4 datasets, identifying the unique challenges of reconstructing both observations and targets of time series data.
We then propose TS-Inverse, a novel gradient inversion attack that improves the inversion of time series data through (i) learning a gradient inversion model that outputs quantile predictions, (ii) a unique loss function incorporating periodicity and trend regularization, and (iii) regularization according to the quantile predictions. Our evaluations demonstrate a remarkable performance of TS-Inverse, achieving at least 2x-10x improvement in terms of sMAPE metric over existing gradient inversion attacks methods on time series data. ...
Unfortunately, privacy risks in federated learning persist, as servers can potentially reconstruct clients' training data through gradient inversion attacks.
While gradient inversion attacks are demonstrated for image, text and tabular classification tasks, little is known for time series regression tasks.
In this paper, we first conduct an extensive empirical study on inverting time series data across 4 time series forecasting models and 4 datasets, identifying the unique challenges of reconstructing both observations and targets of time series data.
We then propose TS-Inverse, a novel gradient inversion attack that improves the inversion of time series data through (i) learning a gradient inversion model that outputs quantile predictions, (ii) a unique loss function incorporating periodicity and trend regularization, and (iii) regularization according to the quantile predictions. Our evaluations demonstrate a remarkable performance of TS-Inverse, achieving at least 2x-10x improvement in terms of sMAPE metric over existing gradient inversion attacks methods on time series data. ...
Federated learning for time series forecasting enables clients with privacy-sensitive time series data to collaboratively learn accurate forecasting models, e.g., in energy load prediction.
Unfortunately, privacy risks in federated learning persist, as servers can potentially reconstruct clients' training data through gradient inversion attacks.
While gradient inversion attacks are demonstrated for image, text and tabular classification tasks, little is known for time series regression tasks.
In this paper, we first conduct an extensive empirical study on inverting time series data across 4 time series forecasting models and 4 datasets, identifying the unique challenges of reconstructing both observations and targets of time series data.
We then propose TS-Inverse, a novel gradient inversion attack that improves the inversion of time series data through (i) learning a gradient inversion model that outputs quantile predictions, (ii) a unique loss function incorporating periodicity and trend regularization, and (iii) regularization according to the quantile predictions. Our evaluations demonstrate a remarkable performance of TS-Inverse, achieving at least 2x-10x improvement in terms of sMAPE metric over existing gradient inversion attacks methods on time series data.
Unfortunately, privacy risks in federated learning persist, as servers can potentially reconstruct clients' training data through gradient inversion attacks.
While gradient inversion attacks are demonstrated for image, text and tabular classification tasks, little is known for time series regression tasks.
In this paper, we first conduct an extensive empirical study on inverting time series data across 4 time series forecasting models and 4 datasets, identifying the unique challenges of reconstructing both observations and targets of time series data.
We then propose TS-Inverse, a novel gradient inversion attack that improves the inversion of time series data through (i) learning a gradient inversion model that outputs quantile predictions, (ii) a unique loss function incorporating periodicity and trend regularization, and (iii) regularization according to the quantile predictions. Our evaluations demonstrate a remarkable performance of TS-Inverse, achieving at least 2x-10x improvement in terms of sMAPE metric over existing gradient inversion attacks methods on time series data.
Watermarking Diffusion Graph Models
GUISE: Graph GaUssIan Shading watErmark
In the expanding field of generative artificial intelligence, the integration of robust watermarking technologies is essential to protect intellectual property and maintain content authenticity. Traditionally, watermarking techniques have been developed primarily for rich information media such as images and audio. However, these methods have not been adequately adapted for graph-based data, particularly on molecular graphs. Latent 3D graph diffusion(LDM-3DG) is an ascendant approach in the molecular graph generation field. This model effectively manages the complexities of molecular structures, preserving essential symmetries and topological features. To protect this sophisticated new technology, we adapt the Gaussian Shading, a proven performance lossless watermarking technique, to the latent graph diffusion domain. Our adaptation simplifies the watermark diffusion process through duplication and padding, making it adaptable and suitable for various message types.
We conduct several experiments using the LDM-3DG model on publicly available datasets QM9 and Drugs, to assess the robustness and effectiveness of our technique. Our results demonstrate that the watermarked molecules maintain statistical parity in 9 out of 10 performance metrics compared to the original. Moreover, they exhibit a 100\% detection rate and a 99\% extraction rate in a 2D decoded pipeline, while also showing robustness against post-editing attacks. ...
We conduct several experiments using the LDM-3DG model on publicly available datasets QM9 and Drugs, to assess the robustness and effectiveness of our technique. Our results demonstrate that the watermarked molecules maintain statistical parity in 9 out of 10 performance metrics compared to the original. Moreover, they exhibit a 100\% detection rate and a 99\% extraction rate in a 2D decoded pipeline, while also showing robustness against post-editing attacks. ...
In the expanding field of generative artificial intelligence, the integration of robust watermarking technologies is essential to protect intellectual property and maintain content authenticity. Traditionally, watermarking techniques have been developed primarily for rich information media such as images and audio. However, these methods have not been adequately adapted for graph-based data, particularly on molecular graphs. Latent 3D graph diffusion(LDM-3DG) is an ascendant approach in the molecular graph generation field. This model effectively manages the complexities of molecular structures, preserving essential symmetries and topological features. To protect this sophisticated new technology, we adapt the Gaussian Shading, a proven performance lossless watermarking technique, to the latent graph diffusion domain. Our adaptation simplifies the watermark diffusion process through duplication and padding, making it adaptable and suitable for various message types.
We conduct several experiments using the LDM-3DG model on publicly available datasets QM9 and Drugs, to assess the robustness and effectiveness of our technique. Our results demonstrate that the watermarked molecules maintain statistical parity in 9 out of 10 performance metrics compared to the original. Moreover, they exhibit a 100\% detection rate and a 99\% extraction rate in a 2D decoded pipeline, while also showing robustness against post-editing attacks.
We conduct several experiments using the LDM-3DG model on publicly available datasets QM9 and Drugs, to assess the robustness and effectiveness of our technique. Our results demonstrate that the watermarked molecules maintain statistical parity in 9 out of 10 performance metrics compared to the original. Moreover, they exhibit a 100\% detection rate and a 99\% extraction rate in a 2D decoded pipeline, while also showing robustness against post-editing attacks.
Time's Up!
Robust Watermarking in Large Language Models for Time Series Generation
The advent of pretrained probabilistic time series foundation models has significantly advanced the field of time series forecasting. Despite these models’ growing popularity, the application of watermarking techniques to them remains underexplored. This paper addresses this research gap by benchmarking several widely used watermarking methods to time series models and by introducing a novel watermarking technique named HTW (Heads Tails Watermark). Unlike traditional probabilistic watermarking approaches, HTW uses a pseudo-random function to directly embed a signal into the numeric structure of the series, thereby greatly enhancing its robustness against potential attacks. Comprehensive experiments and evaluations reveal that on average, HTW retains 98.4% prediction accuracy, significantly outperforming other conventional LLM watermarks. Furthermore, HTW demonstrates robust performance with an average z-score of 5.28 across various datasets and attack scenarios for a series length of 48. These findings establish HTW as a superior alternative for securing pretrained probabilistic time series foundation models
...
The advent of pretrained probabilistic time series foundation models has significantly advanced the field of time series forecasting. Despite these models’ growing popularity, the application of watermarking techniques to them remains underexplored. This paper addresses this research gap by benchmarking several widely used watermarking methods to time series models and by introducing a novel watermarking technique named HTW (Heads Tails Watermark). Unlike traditional probabilistic watermarking approaches, HTW uses a pseudo-random function to directly embed a signal into the numeric structure of the series, thereby greatly enhancing its robustness against potential attacks. Comprehensive experiments and evaluations reveal that on average, HTW retains 98.4% prediction accuracy, significantly outperforming other conventional LLM watermarks. Furthermore, HTW demonstrates robust performance with an average z-score of 5.28 across various datasets and attack scenarios for a series length of 48. These findings establish HTW as a superior alternative for securing pretrained probabilistic time series foundation models
Tabular data is one of the most common forms of data in the industry and science. Recent research on synthetic data generation employs auto-regressive generative large language models (LLMs) to create highly realistic tabular data samples. With the increasing use of LLMs, there is a need to govern the data generated by these models, for instance, by watermarking the model output. While the state-of-the-art Soft Red List watermarking framework has shown impressive results on standard language models, it can not be seamlessly applied to models fine-tuned for generating tabular data due to i) column permutation and ii) the task’s nature of generating low entropy sequences. We propose Tabular Red GrEen LiST (T-REST), an adaptation of the Soft Red List watermarking algorithm on tabular LLMs that is agnostic to column permutation and improves detection efficiency by employing a weighted count method that favors columns with higher entropy. Our experiments on 4 real-world datasets demonstrate that T-REST introduces a nonsignificant drop of 3% in the synthetic data quality compared to the non-watermarked data, using the resemblance and downstream machine learning efficiency metrics, while achieving high detection accuracy with AUROC of over 0.98. T-REST is insusceptible to any column or row permutation and is robust against post-editing attacks on categorical columns by maintaining a True Positive Rate (TPR) of over 0.85 when 50% of categorical values are modified.
...
Tabular data is one of the most common forms of data in the industry and science. Recent research on synthetic data generation employs auto-regressive generative large language models (LLMs) to create highly realistic tabular data samples. With the increasing use of LLMs, there is a need to govern the data generated by these models, for instance, by watermarking the model output. While the state-of-the-art Soft Red List watermarking framework has shown impressive results on standard language models, it can not be seamlessly applied to models fine-tuned for generating tabular data due to i) column permutation and ii) the task’s nature of generating low entropy sequences. We propose Tabular Red GrEen LiST (T-REST), an adaptation of the Soft Red List watermarking algorithm on tabular LLMs that is agnostic to column permutation and improves detection efficiency by employing a weighted count method that favors columns with higher entropy. Our experiments on 4 real-world datasets demonstrate that T-REST introduces a nonsignificant drop of 3% in the synthetic data quality compared to the non-watermarked data, using the resemblance and downstream machine learning efficiency metrics, while achieving high detection accuracy with AUROC of over 0.98. T-REST is insusceptible to any column or row permutation and is robust against post-editing attacks on categorical columns by maintaining a True Positive Rate (TPR) of over 0.85 when 50% of categorical values are modified.
Data in the form of tables is commonly used in the scientific and research industry, as it provides a compact, easy-to-understand and logical way of storing data. The advancement of diffusion models has significantly improved the quality of generated tabular data, but it also poses risks of misappropriation and copyright concerns. Thus, there is a need to control and monitor the data generated by diffusion models, to enable harm mitigation and protect intellectual property. This paper addresses the necessity for robust watermarking techniques specifically designed for tabular data generated by diffusion models. We propose Ellipse, a generalization of the Tree-Ring watermarking method, originally developed for square-shaped images, to handle rectangular shaped tables. We change the shape of the watermark from a circle---fit for square-shaped images---to an oval---fit for datasets of rectangle shape.
Through comprehensive experiments on four real world datasets (Abalone, Adult, Default, and Diabetes), we demonstrate that the adapted watermarking technique has a negligible drop of 3.5% in data quality, measured through correlations between real and synthetic distributions, performance of downstream machine learning tasks, and discriminability between the real and synthetic data. This is a better result than the 12.46% drop in data quality offered by having a circle mask. Ellipse introduces a non-significant average drop of 0.4% in detection efficiency compared to having a circle mask. Our implementation also offers resilience against value skewing and deletion attacks on the rows and columns of the dataset. When exposed to attacks, Ellipse has a higher Area Under the Curve (AUC) than the circular mask of Tree-Ring by an average of 7.17%. The code for Ellipse is publicly available at https://github.com/6toma/ellipse-watermark.
...
Through comprehensive experiments on four real world datasets (Abalone, Adult, Default, and Diabetes), we demonstrate that the adapted watermarking technique has a negligible drop of 3.5% in data quality, measured through correlations between real and synthetic distributions, performance of downstream machine learning tasks, and discriminability between the real and synthetic data. This is a better result than the 12.46% drop in data quality offered by having a circle mask. Ellipse introduces a non-significant average drop of 0.4% in detection efficiency compared to having a circle mask. Our implementation also offers resilience against value skewing and deletion attacks on the rows and columns of the dataset. When exposed to attacks, Ellipse has a higher Area Under the Curve (AUC) than the circular mask of Tree-Ring by an average of 7.17%. The code for Ellipse is publicly available at https://github.com/6toma/ellipse-watermark.
...
Data in the form of tables is commonly used in the scientific and research industry, as it provides a compact, easy-to-understand and logical way of storing data. The advancement of diffusion models has significantly improved the quality of generated tabular data, but it also poses risks of misappropriation and copyright concerns. Thus, there is a need to control and monitor the data generated by diffusion models, to enable harm mitigation and protect intellectual property. This paper addresses the necessity for robust watermarking techniques specifically designed for tabular data generated by diffusion models. We propose Ellipse, a generalization of the Tree-Ring watermarking method, originally developed for square-shaped images, to handle rectangular shaped tables. We change the shape of the watermark from a circle---fit for square-shaped images---to an oval---fit for datasets of rectangle shape.
Through comprehensive experiments on four real world datasets (Abalone, Adult, Default, and Diabetes), we demonstrate that the adapted watermarking technique has a negligible drop of 3.5% in data quality, measured through correlations between real and synthetic distributions, performance of downstream machine learning tasks, and discriminability between the real and synthetic data. This is a better result than the 12.46% drop in data quality offered by having a circle mask. Ellipse introduces a non-significant average drop of 0.4% in detection efficiency compared to having a circle mask. Our implementation also offers resilience against value skewing and deletion attacks on the rows and columns of the dataset. When exposed to attacks, Ellipse has a higher Area Under the Curve (AUC) than the circular mask of Tree-Ring by an average of 7.17%. The code for Ellipse is publicly available at https://github.com/6toma/ellipse-watermark.
Through comprehensive experiments on four real world datasets (Abalone, Adult, Default, and Diabetes), we demonstrate that the adapted watermarking technique has a negligible drop of 3.5% in data quality, measured through correlations between real and synthetic distributions, performance of downstream machine learning tasks, and discriminability between the real and synthetic data. This is a better result than the 12.46% drop in data quality offered by having a circle mask. Ellipse introduces a non-significant average drop of 0.4% in detection efficiency compared to having a circle mask. Our implementation also offers resilience against value skewing and deletion attacks on the rows and columns of the dataset. When exposed to attacks, Ellipse has a higher Area Under the Curve (AUC) than the circular mask of Tree-Ring by an average of 7.17%. The code for Ellipse is publicly available at https://github.com/6toma/ellipse-watermark.
Go With The Flow: Fault-Tolerant Decentralized Training of Large Language Models
Decentralised Training of Large Language Models
Motivated by the emergence of Large Language Models (LLMs) and the importance of democratizing their training, we propose Go With The Flow, the first practical decentralized training framework for LLMs. Differently from existing distributed and federated training frameworks, Go With The Flow enables the collaborative training of an LLM on a set of heterogeneous client nodes that dedicate different resources for an undefined amount of time. Our work addresses node churn, i.e., clients joining or leaving the system, and network instabilities, i.e., network links becoming unstable or unreliable. The core of Go With The Flow is a decentralized flow algorithm that finds the most effective routing to train a maximum number of microbatches with a minimum delay. We extensively evaluate our work on LLama-like and GPT-like models, compare it against the prior art and achieve up to 45\% training time reduction in realistic and challenging scenarios of heterogeneous client nodes distributed at 10 different geographic locations with a high node churn rate. We further demonstrate resilient training in such challenging environments, without sacrificing convergence.
...
Motivated by the emergence of Large Language Models (LLMs) and the importance of democratizing their training, we propose Go With The Flow, the first practical decentralized training framework for LLMs. Differently from existing distributed and federated training frameworks, Go With The Flow enables the collaborative training of an LLM on a set of heterogeneous client nodes that dedicate different resources for an undefined amount of time. Our work addresses node churn, i.e., clients joining or leaving the system, and network instabilities, i.e., network links becoming unstable or unreliable. The core of Go With The Flow is a decentralized flow algorithm that finds the most effective routing to train a maximum number of microbatches with a minimum delay. We extensively evaluate our work on LLama-like and GPT-like models, compare it against the prior art and achieve up to 45\% training time reduction in realistic and challenging scenarios of heterogeneous client nodes distributed at 10 different geographic locations with a high node churn rate. We further demonstrate resilient training in such challenging environments, without sacrificing convergence.
In many scientific fields, time series data is essen- tial, yet maintaining the integrity and legitimacy of such data is still difficult. Traditional watermarking techniques have mainly been used for multimedia. Although approaches for watermarking non-media data have been developed recently, there is still a big gap in the development of reliable and undetectable watermarking methods for time series diffusion models. We suggest a novel modification of the tree ring watermarking method for the 2D time series model LDCast, which is intended for precipitation prediction.
Through the incorporation of watermarks into the model’s process, we guarantee resilience and undetectability. Our approach preserves the LDCast model’s predicted accuracy while still being able to verifying the origins of the data. We confirm the efficacy of our method through comprehensive evaluation, underscoring its potential to improve the se- curity and integrity of time series forecasting models. ...
Through the incorporation of watermarks into the model’s process, we guarantee resilience and undetectability. Our approach preserves the LDCast model’s predicted accuracy while still being able to verifying the origins of the data. We confirm the efficacy of our method through comprehensive evaluation, underscoring its potential to improve the se- curity and integrity of time series forecasting models. ...
In many scientific fields, time series data is essen- tial, yet maintaining the integrity and legitimacy of such data is still difficult. Traditional watermarking techniques have mainly been used for multimedia. Although approaches for watermarking non-media data have been developed recently, there is still a big gap in the development of reliable and undetectable watermarking methods for time series diffusion models. We suggest a novel modification of the tree ring watermarking method for the 2D time series model LDCast, which is intended for precipitation prediction.
Through the incorporation of watermarks into the model’s process, we guarantee resilience and undetectability. Our approach preserves the LDCast model’s predicted accuracy while still being able to verifying the origins of the data. We confirm the efficacy of our method through comprehensive evaluation, underscoring its potential to improve the se- curity and integrity of time series forecasting models.
Through the incorporation of watermarks into the model’s process, we guarantee resilience and undetectability. Our approach preserves the LDCast model’s predicted accuracy while still being able to verifying the origins of the data. We confirm the efficacy of our method through comprehensive evaluation, underscoring its potential to improve the se- curity and integrity of time series forecasting models.
Synthetic tabular data generated by tabular generative models represent an effective means of augmenting and sharing data. It is of paramount importance to trace and audit such synthetic data, avoiding potential harms and risks associated with inappropriate usage. While watermarking techniques are increasingly used for synthetic images, little is known about how to watermark synthetic tables such that they are imperceptible for humans, detectable by algorithms, and robust against post-editing. In this paper, we present the first watermarking algorithm for tabular diffusion models, which inserts novel ripple watermarks into the latent space of tables. For every synthetic table, the watermark initiates from a central ring within
the Fourier-transformed latent of the table, extending gradually across a large portion of the space. The watermark can be detected by calculating the distance between the Fourier-transformed tabular latent and the ground-truth watermark patch. Additionally, we develop post-editing attacks, including row/column/value deletion and distortion, to evaluate the robustness of the watermark. Our evaluation on four datasets demonstrates that our watermarking scheme effectively preserves the quality of synthetic tables in terms of resemblance, discriminability, and downstream utility. The average quality difference is less than 0.6% compared to non-watermarked data, while maintaining high detectability, with average statistical p-values over 25× lower than 0.02. Additionally, our robustness analysis
shows that the watermark is resilient against various post-editing actions, with
85% of the p-values remaining below 0.05 across all 18 attack settings on four
datasets. ...
the Fourier-transformed latent of the table, extending gradually across a large portion of the space. The watermark can be detected by calculating the distance between the Fourier-transformed tabular latent and the ground-truth watermark patch. Additionally, we develop post-editing attacks, including row/column/value deletion and distortion, to evaluate the robustness of the watermark. Our evaluation on four datasets demonstrates that our watermarking scheme effectively preserves the quality of synthetic tables in terms of resemblance, discriminability, and downstream utility. The average quality difference is less than 0.6% compared to non-watermarked data, while maintaining high detectability, with average statistical p-values over 25× lower than 0.02. Additionally, our robustness analysis
shows that the watermark is resilient against various post-editing actions, with
85% of the p-values remaining below 0.05 across all 18 attack settings on four
datasets. ...
Synthetic tabular data generated by tabular generative models represent an effective means of augmenting and sharing data. It is of paramount importance to trace and audit such synthetic data, avoiding potential harms and risks associated with inappropriate usage. While watermarking techniques are increasingly used for synthetic images, little is known about how to watermark synthetic tables such that they are imperceptible for humans, detectable by algorithms, and robust against post-editing. In this paper, we present the first watermarking algorithm for tabular diffusion models, which inserts novel ripple watermarks into the latent space of tables. For every synthetic table, the watermark initiates from a central ring within
the Fourier-transformed latent of the table, extending gradually across a large portion of the space. The watermark can be detected by calculating the distance between the Fourier-transformed tabular latent and the ground-truth watermark patch. Additionally, we develop post-editing attacks, including row/column/value deletion and distortion, to evaluate the robustness of the watermark. Our evaluation on four datasets demonstrates that our watermarking scheme effectively preserves the quality of synthetic tables in terms of resemblance, discriminability, and downstream utility. The average quality difference is less than 0.6% compared to non-watermarked data, while maintaining high detectability, with average statistical p-values over 25× lower than 0.02. Additionally, our robustness analysis
shows that the watermark is resilient against various post-editing actions, with
85% of the p-values remaining below 0.05 across all 18 attack settings on four
datasets.
the Fourier-transformed latent of the table, extending gradually across a large portion of the space. The watermark can be detected by calculating the distance between the Fourier-transformed tabular latent and the ground-truth watermark patch. Additionally, we develop post-editing attacks, including row/column/value deletion and distortion, to evaluate the robustness of the watermark. Our evaluation on four datasets demonstrates that our watermarking scheme effectively preserves the quality of synthetic tables in terms of resemblance, discriminability, and downstream utility. The average quality difference is less than 0.6% compared to non-watermarked data, while maintaining high detectability, with average statistical p-values over 25× lower than 0.02. Additionally, our robustness analysis
shows that the watermark is resilient against various post-editing actions, with
85% of the p-values remaining below 0.05 across all 18 attack settings on four
datasets.
Abstract— Federated Learning (FL) makes it possible for a network of clients to jointly train a machine learning model, while also keeping the training data private. There are several approaches when designing a FL network and while most existing research is focused on a single-server design, new and promising variations are arising that make use of multiple servers, witch have the benefit of speeding up the training process. Unfortunately single-server FL networks are prone to model poisoning attacks by malicious participants, that aim to reduce the accuracy of the trained model. This work showcases the inherent resilience of the multi-server design against existing state-of-the-art attacks tailored around single-server FL, as well as propose two novel attacks that exploits multi-server topology in order to reduce the required knowledge an adversary needs to obtain to carry out the attack, while still remaining effective. Main findings are as follows: In the event that the malicious party has compromised the entire network, existing single-server attacks are sufficient to completely prevent a model from training. If they are limited to knowledge available only within the local reach of their compromised clients, the effect is minimized to where the attacks might get mitigated without any defences being necessary. However in such cases a correlation can be observed between the location of the compromised clients and the effectiveness of an attack. The novel attacks proposed in this paper exploit this relation in order to remain sufficiently effective while requiring only the same amount of data necessary for the multi-server algorithm to function.
...
Abstract— Federated Learning (FL) makes it possible for a network of clients to jointly train a machine learning model, while also keeping the training data private. There are several approaches when designing a FL network and while most existing research is focused on a single-server design, new and promising variations are arising that make use of multiple servers, witch have the benefit of speeding up the training process. Unfortunately single-server FL networks are prone to model poisoning attacks by malicious participants, that aim to reduce the accuracy of the trained model. This work showcases the inherent resilience of the multi-server design against existing state-of-the-art attacks tailored around single-server FL, as well as propose two novel attacks that exploits multi-server topology in order to reduce the required knowledge an adversary needs to obtain to carry out the attack, while still remaining effective. Main findings are as follows: In the event that the malicious party has compromised the entire network, existing single-server attacks are sufficient to completely prevent a model from training. If they are limited to knowledge available only within the local reach of their compromised clients, the effect is minimized to where the attacks might get mitigated without any defences being necessary. However in such cases a correlation can be observed between the location of the compromised clients and the effectiveness of an attack. The novel attacks proposed in this paper exploit this relation in order to remain sufficiently effective while requiring only the same amount of data necessary for the multi-server algorithm to function.
Exploring the Impact of Single-Character Attacks in Federated Learning Language Classification
Introducing the Novel Single-Character Strike
Federated learning (FL) is a privacy preserving machine learning approach which allows a machine learning model to be trained in a distributed fashion without ever sharing user data. Due to the large amount of valuable text and voice data stored on end-user devices, this approach works particularly well for natural language processing (NLP) tasks. Due to many applications making use of the algorithm and increasing interest in academics, ensuring security is essential. Current backdoor attacks in NLP tasks are still unable to evade some defence mechanisms. Therefore, we propose a novel attack, the single-character strike to address this research gap. Consequently, the following research question is posed: What are the properties of the single-character strike in a language classification task? By experimental analysis the following properties are discovered: the single-character strike is undetectable against five state-of-the-art defences, has low impact on the global model accuracy, trains slower than similar attacks, relies on characters on the edge of the distribution to function, is robust within the global model, and performs best when close to convergence and with more adversarial clients. Emphasizing its imperceptibility and persistence, the attack maintains a 70\% backdoor accuracy after a thousand iterations without training and remains undetectable against: (Multi-)Krum, RFA, Norm Clipping and Weak Differential Privacy. By providing insight into the effective single-character strike, this paper adds to the growing body of work that questions whether federated learning can be secure against backdoor attacks.
...
Federated learning (FL) is a privacy preserving machine learning approach which allows a machine learning model to be trained in a distributed fashion without ever sharing user data. Due to the large amount of valuable text and voice data stored on end-user devices, this approach works particularly well for natural language processing (NLP) tasks. Due to many applications making use of the algorithm and increasing interest in academics, ensuring security is essential. Current backdoor attacks in NLP tasks are still unable to evade some defence mechanisms. Therefore, we propose a novel attack, the single-character strike to address this research gap. Consequently, the following research question is posed: What are the properties of the single-character strike in a language classification task? By experimental analysis the following properties are discovered: the single-character strike is undetectable against five state-of-the-art defences, has low impact on the global model accuracy, trains slower than similar attacks, relies on characters on the edge of the distribution to function, is robust within the global model, and performs best when close to convergence and with more adversarial clients. Emphasizing its imperceptibility and persistence, the attack maintains a 70\% backdoor accuracy after a thousand iterations without training and remains undetectable against: (Multi-)Krum, RFA, Norm Clipping and Weak Differential Privacy. By providing insight into the effective single-character strike, this paper adds to the growing body of work that questions whether federated learning can be secure against backdoor attacks.
Federated learning provides a lot of opportunities, especially with the built-in privacy considerations. There is however one attack that might compromise the utility of federated learning: backdoor attacks [14]. There are already some existing defenses, like flame [13] but they are computationally expensive [14]. This paper evaluates a version of differential privacy, where the Gaussian noise added to the aggravated model of the clipped updates is smaller than usually. This is often referred to as weak differential privacy or weakDP. This paper evaluates weakDP with different parameters to find if weakDP can be used as a defense for a language processing federated learning classifier against a backdoor attack.
...
Federated learning provides a lot of opportunities, especially with the built-in privacy considerations. There is however one attack that might compromise the utility of federated learning: backdoor attacks [14]. There are already some existing defenses, like flame [13] but they are computationally expensive [14]. This paper evaluates a version of differential privacy, where the Gaussian noise added to the aggravated model of the clipped updates is smaller than usually. This is often referred to as weak differential privacy or weakDP. This paper evaluates weakDP with different parameters to find if weakDP can be used as a defense for a language processing federated learning classifier against a backdoor attack.
Time-Series Forecasting with Hybrid Federated Learning
A Personalized Approach to Collaboration
Collaborative efforts in Predictive Maintenance and Control can be beneficial for manufacturers and customers in industrial environments. However, these efforts are challenged by the need for multi-dimensional sharing of information about the same type (horizontal) and piece (vertical) of equipment, privacy restrictions and the presence of heterogeneous data distributions across participants.
Existing solutions have addressed some of these challenges for forecasting or different purposes but there lacks a comprehensive approach that handles all of them for time series forecasting. To solve this problem, we introduce Time-series-based Personalized Hybrid Federated Learning (TPHFL), a hybrid federated learning (FL) strategy that combines Horizontal FL and Vertical FL to enable multi-level knowledge exchange while preserving data privacy. All participants use a personalization mechanism to make predictions that better suit their underlying data distribution.
Our approach employs a distributed model to handle vertical privacy constraints and addresses data heterogeneity across equipment through a personalisation mechanism. Through extensive experiments on four public and one industry-specific datasets, we show that TPHFL outperforms independent learning scenarios by 27.2%, providing a strong incentive for parties to collaborate.
We demonstrate the effectiveness of personalisation by showing an accuracy improvement of up to 42.7% when comparing TPHFL with personalisation to TPHFL without personalisation, and 32.7% when comparing traditional HFL methods to HFL with personalisation. Additionally, we evaluate a different configuration for personalisation and perform a detailed hyperparameter analysis to better understand the behaviour of TPHFL in different contexts. ...
Existing solutions have addressed some of these challenges for forecasting or different purposes but there lacks a comprehensive approach that handles all of them for time series forecasting. To solve this problem, we introduce Time-series-based Personalized Hybrid Federated Learning (TPHFL), a hybrid federated learning (FL) strategy that combines Horizontal FL and Vertical FL to enable multi-level knowledge exchange while preserving data privacy. All participants use a personalization mechanism to make predictions that better suit their underlying data distribution.
Our approach employs a distributed model to handle vertical privacy constraints and addresses data heterogeneity across equipment through a personalisation mechanism. Through extensive experiments on four public and one industry-specific datasets, we show that TPHFL outperforms independent learning scenarios by 27.2%, providing a strong incentive for parties to collaborate.
We demonstrate the effectiveness of personalisation by showing an accuracy improvement of up to 42.7% when comparing TPHFL with personalisation to TPHFL without personalisation, and 32.7% when comparing traditional HFL methods to HFL with personalisation. Additionally, we evaluate a different configuration for personalisation and perform a detailed hyperparameter analysis to better understand the behaviour of TPHFL in different contexts. ...
Collaborative efforts in Predictive Maintenance and Control can be beneficial for manufacturers and customers in industrial environments. However, these efforts are challenged by the need for multi-dimensional sharing of information about the same type (horizontal) and piece (vertical) of equipment, privacy restrictions and the presence of heterogeneous data distributions across participants.
Existing solutions have addressed some of these challenges for forecasting or different purposes but there lacks a comprehensive approach that handles all of them for time series forecasting. To solve this problem, we introduce Time-series-based Personalized Hybrid Federated Learning (TPHFL), a hybrid federated learning (FL) strategy that combines Horizontal FL and Vertical FL to enable multi-level knowledge exchange while preserving data privacy. All participants use a personalization mechanism to make predictions that better suit their underlying data distribution.
Our approach employs a distributed model to handle vertical privacy constraints and addresses data heterogeneity across equipment through a personalisation mechanism. Through extensive experiments on four public and one industry-specific datasets, we show that TPHFL outperforms independent learning scenarios by 27.2%, providing a strong incentive for parties to collaborate.
We demonstrate the effectiveness of personalisation by showing an accuracy improvement of up to 42.7% when comparing TPHFL with personalisation to TPHFL without personalisation, and 32.7% when comparing traditional HFL methods to HFL with personalisation. Additionally, we evaluate a different configuration for personalisation and perform a detailed hyperparameter analysis to better understand the behaviour of TPHFL in different contexts.
Existing solutions have addressed some of these challenges for forecasting or different purposes but there lacks a comprehensive approach that handles all of them for time series forecasting. To solve this problem, we introduce Time-series-based Personalized Hybrid Federated Learning (TPHFL), a hybrid federated learning (FL) strategy that combines Horizontal FL and Vertical FL to enable multi-level knowledge exchange while preserving data privacy. All participants use a personalization mechanism to make predictions that better suit their underlying data distribution.
Our approach employs a distributed model to handle vertical privacy constraints and addresses data heterogeneity across equipment through a personalisation mechanism. Through extensive experiments on four public and one industry-specific datasets, we show that TPHFL outperforms independent learning scenarios by 27.2%, providing a strong incentive for parties to collaborate.
We demonstrate the effectiveness of personalisation by showing an accuracy improvement of up to 42.7% when comparing TPHFL with personalisation to TPHFL without personalisation, and 32.7% when comparing traditional HFL methods to HFL with personalisation. Additionally, we evaluate a different configuration for personalisation and perform a detailed hyperparameter analysis to better understand the behaviour of TPHFL in different contexts.