Google DP vs. OpenDP: Empirical Comparison of Differential Privacy Libraries

This research looks at two open-source tools for differential privacy: Google's Differential Privacy Library and the OpenDP Library. The main aim of this study is to test them side-by-side and observe how they compared quantitatively. Specifically, the focus is on their implementations of the Laplace noise mechanism. It measured their computational performance, scalability and the accuracy of their results (utility). These experiments were done for basic Count and Sum queries on synthetic structured datasets, using different privacy settings (ϵ) and dataset sizes.

The experiments using the Laplace mechanism showed some clear differences. GoogleDP was consistently faster than OpenDP for both Count and Sum queries and this speed advantage was especially noticeable with larger datasets. The choice of ϵ did not really change how long either library took to run. When it came to utility, both libraries showed the expected pattern: more privacy (lower ϵ) meant less accurate results. The accuracy differences between the two were small. For Sum queries OpenDP gave results with slightly better utility, especially on large datasets and when ϵ was small (meaning more privacy). This paper offers current, data-supported information about the practical pros and cons when using these libraries for these common DP operations with the Laplace mechanism. A full comparison including the Gaussian mechanism could not be completed in this study due to the fact that PyDP (GoogleDP's python wrapper) had only the Laplace mechanism implemented. This remains an important task for future work.