Contrastive Self-Supervised Learning for Utility-Privacy Trade-off Navigation in Graph Neural Networks

Abstract

Graphs are everywhere in the real world. Although Graph Neural Networks (GNNs) have demonstrated strong performance across a wide range of graph-based tasks, their unique neighbourhood aggregation mechanism encodes sensitive feature, label, and topological information into model weights, making them vulnerable to Membership Inference Attacks (MIA). This poses challenges for organizations regarding the practical deployment of GNN models trained on their private data, as they have the obligation to protect the privacy of individuals in the dataset. While numerous privacy-preserving techniques, including differential privacy (DP) based approaches such as PrivGNN, have been proposed, they are subject to the inherent utility-privacy trade-off where achieving strong privacy often comes at the cost of reduction in model utility. In this work, we investigate Self-Supervised Learning (SSL) pretraining on a public graph, with a focus on contrastive pretraining via GRACE, as a utility-preserving privacy mitigation strategy with the potential to improve privacy without sacrificing generalizability. All experiments are performed under an inductive setting with disjoint graphs, where the model is evaluated on nodes never seen during training. We first show that SSL improves the utility-privacy trade-off in standard GNN training, with the backbone learning rate (BLR) acting as a tunable knob to navigate the trade-off. Subsequently, we perform a node-level vulnerability analysis, finding that structural isolation and intra-class feature dissimilarity act as predictors for nodes' privacy vulnerability, while neighbourhood class divergence exhibits more dataset-dependent results. We then show that employing lower BLR alone can disproportionately reduce the privacy vulnerability of the most exposed nodes, with SSL providing additional, uniform privacy benefit on top of it. Beyond standard GNN training, we integrate SSL into DP-based PrivGNN and show that it substantially recovers the utility lost to privacy mechanisms, providing significantly tighter formal privacy guarantees without hurting empirical privacy.