Jack Fitzsimons, Oblivious
Differential privacy (DP) implementations are notoriously prone to bugs. Small mistakes can compromise or completely remove the protection provided to user data. And developers building them have few tools to help them get it right. While there are some ways to test DP tools, their limitations mean that, in practice, developers are reduced to trying very hard not to make mistakes.
To help developers build better DP tools, we introduce a new open-source test framework that can detect many common implementation bugs. We explain how it works and what types of problems it can identify. And we show that it works in practice by using it to identify 13 bugs in 11 open-source differential privacy libraries, and discussing how it helped us build the latest version of our DP SQL engine.
This talk is adapted from a PoPETS paper by my colleagues Tudor Cebere, David Erb, Damien Desfontaines, Aurélien Bellet, and Jack Fitzsimons.
Jack is the CTO and co-founder of Oblivious, an Irish-based technology company focused on privacy-enhancing technologies. He holds a D.Phil (PhD) from the University of Oxford, and has worked on a wide range of data-centric challenges in industry; from topics in computer vision at NASA's Jet Propulsion Laboratory to quantitative data analysis at ElectroRoute, the European energy trading subsidiary of Mitsubishi. Jack has been an active member of the UN's Privacy-Preserving Technologies Task Team since 2020, the UN PET Lab since its inception and an author of the NIST Differential Privacy Deployment Registry.
author = {Jack Fitzsimons},
title = {Privacy in Theory, Bugs in Practice: {Grey-Box} Testing for Differential Privacy Libraries},
year = {2026},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = jun
}