When Machine Learning Models Leak

When Machine Learning Models Leak: An Exploration of Synthetic Training Data

Slokom, M. (TU Delft Multimedia Computing; Statistics Netherlands (CBS); Radboud Universiteit Nijmegen)
de Wolf, Peter Paul (Statistics Netherlands (CBS))
Larson, M.A. (TU Delft Multimedia Computing; Radboud Universiteit Nijmegen)

Domingo-Ferrer, Josep (editor)
Laurent, Maryline (editor)

2022

We investigate an attack on a machine learning classifier that predicts the propensity of a person or household to move (i.e., relocate) in the next two years. The attack assumes that the classifier has been made publically available and that the attacker has access to information about a certain number of target individuals. That attacker might also have information about another set of people to train an auxiliary classifier. We show that the attack is possible for target individuals independently of whether they were contained in the original training set of the classifier. However, the attack is somewhat less successful for individuals that were not contained in the original data. Based on this observation, we investigate whether training the classifier on a data set that is synthesized from the original training data, rather than using the original training data directly, would help to mitigate the effectiveness of the attack. Our experimental results show that it does not, leading us to conclude that new approaches to data synthesis must be developed if synthesized data is to resemble “unseen” individuals to an extent great enough to help to block machine learning model attacks.

Attribute inference
Machine learning
Propensity to move
Synthetic data

http://resolver.tudelft.nl/uuid:c6a83f5c-1808-4cd2-b725-e7b474ab54ff

https://doi.org/10.1007/978-3-031-13945-1_20

Springer

2023-07-01

9783031139444

Privacy in Statistical Databases - International Conference, PSD 2022, Proceedings

International Conference on Privacy in Statistical Databases, PSD 2022, 2022-09-21 → 2022-09-23, Paris, France

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 0302-9743, 13463 LNCS

Green Open Access added to TU Delft Institutional Repository 'You share, we take care!' - Taverne project https://www.openaccess.nl/en/you-share-we-take-care Otherwise as indicated in the copyright section: the publisher is the copyright holder of this work and the author uses the Dutch legislation to make this work public.

Institutional Repository

conference paper

© 2022 M. Slokom, Peter Paul de Wolf, M.A. Larson