PEPR '26 Conference Program

Differentially Private (DP) synthetic data is a promising solution for enabling data-driven innovation while protecting user privacy. However, transforming cutting-edge research in DP into robust, scalable, and usable production systems presents significant engineering challenges. Our library, DPSynth, is based on state-of-the-art marginal-based mechanisms (McKenna et al., 2022), and builds upon the foundations of PipelineDP and mbi libraries.

This talk will share our experience in building and applying DPSynth in production settings, highlighting the journey of productionalizing these research concepts. We'll discuss how DPSynth is built to scale for massive datasets using technologies like Apache Beam and Apache Spark. We will also cover key engineering aspects such as handling real-world data constraints to ensure synthetic data utility and validity, and designing for usability with reasonable defaults for non-DP experts. The library is slated for open-source release prior to the conference, aiming to foster wider adoption of practical DP synthetic data techniques.

Authors: Ryan McKenna, Peter Kairouz, Alexander Knop, Vadym Doroshenko, Eva Bertels

LLM-based simulators are promising new tools for generating synthetic data, especially under conditions that are challenging for traditional Differential Privacy (DP) methods (i.e., high-dimensional tabular data). By condensing customer attributes and behaviors into compact user profiles under DP, we can seed an LLM with data to generate realistic customer transactions. We tested this "Profile-then-Simulate" approach on financial transaction data using PersonaLedger, an LLM-based generator, and compared it to direct DP synthesis on the same dataset.

We found that the LLM-based approach produces usable synthetic data, but direct synthesis still significantly outperforms it on both fraud detection utility and distributional fidelity. We identified systematic LLM biases, not DP noise, as the dominant source of error. The model's learned priors about "typical" financial behavior consistently overrode the statistical distributions we provided as input, particularly for demographic and categorical features, resulting in divergent output data.

This talk shares practical lessons for privacy engineers considering generative AI for synthetic data: (1) LLM biases may dominate DP noise as the primary source of distributional error; (2) direct DP synthesis remains competitive for tractable datasets; and (3) rigorous fidelity evaluation is essential before deploying LLM-generated synthetic data in production pipelines.

Coauthors: Dehao Yuan, Nam H. Nguyen, Mayana Pereira

AI has transformed how people learn, work and live - automating complex tasks and extracting insight from massive datasets. But most powerful AI today (especially large language models) runs on server-class hardware, which typically means user prompts and context must be visible to the service provider to be processed. While acceptable for some cases, it is still challenging with highly sensitive data where users expect similar protections as end-to-end encryption. Private Verifiable Compute (PVC) is a technical solution that can enable users to initiate a request to a private and verifiable environment for context-aware AI processing with sensitive data, where no one, including service providers, can access them. With PVC in the cloud environment, it unleashes full potentials of AI hardware in the data center for complex AI tasks, such as large language models (LLMs), generative AI and beyond, while guaranteeing user privacy and verifiable transparency.

Differential privacy (DP) implementations are notoriously prone to bugs. Small mistakes can compromise or completely remove the protection provided to user data. And developers building them have few tools to help them get it right. While there are some ways to test DP tools, their limitations mean that, in practice, developers are reduced to trying very hard not to make mistakes.

To help developers build better DP tools, we introduce a new open-source test framework that can detect many common implementation bugs. We explain how it works and what types of problems it can identify. And we show that it works in practice by using it to identify 13 bugs in 11 open-source differential privacy libraries, and discussing how it helped us build the latest version of our DP SQL engine.

This talk is adapted from a PoPETS paper by my colleagues Tudor Cebere, David Erb, Damien Desfontaines, Aurélien Bellet, and Jack Fitzsimons.

Many privacy failures are caused less by obvious code that handles privacy such as encryption, retention, deletion, and more by everyday design and coding choices that unintentionally expose sensitive data. A common example is Default Grant Access (DGA), whereby access is permitted unless a developer explicitly blocks it. DGA is difficult for traditional static analysis because the risk often emerges from context: defaults, conditional logic, and framework behavior.

We report lessons learned from deploying language model-based detection for DGA in pull requests at scale, and from replacing a static prompt approach with retrieval-augmented generation (RAG) to address high false-positive rates. In an evaluation spanning nine production repositories, four languages, and 183,000+ methods, the RAG system identified more than twice as many confirmed privacy-relevant issues as the standalone LLM approach. However, false-positive rates did not improve significantly, and we observed language-specific noise, especially in TypeScript and C++.

We cover practical engineering insights: Curating high-quality examples from known incidents, managing corpus retrieval trade-offs and integrating detections into pull request review workflows without overwhelming reviewers.

Non-Human Identity is the fastest-growing attack surface in cloud data environments — and the least discussed. While zero-trust implementations focus on user authentication and network segmentation, the persistent service accounts powering automated pipelines remain long-lived, over-privileged, and largely unexamined. In federal healthcare environments processing genomic and clinical data under FedRAMP High constraints, a single compromised ingestion role means bucket-wide access for up to 90 days. That is not least privilege. That is a liability.

This talk is a production case study, not a proposal. We deployed an Identity-Per-Transaction (IPT) pipeline for a federal life sciences agency, generating a unique cryptographically scoped ephemeral credential for every file ingestion event and destroying it milliseconds later. We present the operational reality of running this in production: STS AssumeRole latency averaging 180ms, peak issuance rates during batch windows, the race condition we hit under high concurrency, and what happens when you need to debug a system whose credentials no longer exist.

We also address two hard problems directly. First, the secret-zero problem: the broker that issues ephemeral credentials is itself a root of trust. If it's compromised, the blast radius is bounded to 900 seconds rather than 90 days, but it is not zero. Second, the genomic privacy boundary: tokenizing 18 HIPAA identifiers does not anonymize a VCF file. This architecture eliminates credential-mediated access risk. It is not a genomic privacy framework. Those are different problems.

Attendees leave with a concrete reference pattern for eliminating Non-Human Identity risk in high-compliance storage systems, an honest accounting of where the architecture holds and where it defers, and open questions about trust bootstrapping that the field has not yet resolved.

We are partnering with the City of Oakland to understand how AI can safely support city workflows and processes. Federal cuts are increasing the strain on local governments, which must meet community needs with fewer resources. Artificial intelligence is often proposed as a solution, but local governments can be wary due to privacy and equity concerns. Oakland is piloting Microsoft Copilot, and we are measuring participant experiences and perceptions throughout the study through interviews and surveys with a sample of participants from a variety of city departments. We will share snapshots of participant perceptions of AI and privacy from multiple touchpoints during this 6 month study. We will also discuss how privacy and equity concerns, including public record requests, disparities in AI access, and fears of skill atrophy, shaped and in some cases, limited participant engagement with AI.

Authors: Shufan Chai, Vinal Dalcy Dsouza, Rebecca Williams Earle, Rasika Bhalerao and Jessica Staddon

Imagine you want your test suite to pass for all commits. Would you create a Test Review process where all teams answer a Tests Questionnaire and submit screenshots as proof, which are then reviewed by Tests Engineers? Obviously, only a maniac would do this: it works, but is much more annoying than a pre-commit hook.

Many companies have some form of Privacy Review. You start with "we'll figure out the requirements", and then ask "did you integrate with deletion tooling?", and before you know it you have screenshots of tests. We can improve privacy and remove manual processes by defining requirements upfront and checking them automatically: you just need to translate engineering language ("write a script that checks all relevant data is delete-able") into compliance language ("mitigate Deletion risks by verifying Key Control Indicators for the Deletion Framework Controls").

We spent the last years at Meta making this change to our Privacy Program: we reduced ad-hoc human decisions in favour of upfront requirements and systematic verification, making our Privacy Reviews both faster and more effective. Out of ~500 requirements, roughly half now rely on automated verifiers. In this talk we'll explain how hopefully you too can do the same!

Additional author: Ben Fonarov

Privacy threat modeling is essential for assessing an application's privacy posture at an architecture level. Yet automated tooling that allows thus analysis to scale remains limited compared to security threat modeling. We present an open-source contribution of twelve privacy threat modeling rules implemented in Threagile, a machine assisted threat modeling toolkit. These rules analyze system architecture, data flows, and technical assets to automatically identify privacy violations including data minimization failures, unauthorized disclosure, insecure storage, and re-identification risks.

Our rules align with the LINDDUN privacy threat modeling framework, covering threat categories such as Linking, Identifying, Data Disclosure, Unawareness, and Non-compliance, while also addressing some OWASP-based Privacy Risks. Each rule provides high fidelity detection logic with actionable mitigations, enabling privacy-by-design practices early in development.

It can be used by privacy architects for assessments as well as designers and developers as a self-service tool. Our contribution bridges the gap between privacy threat modeling theory and practical implementation, offering a systematic privacy risk assessment.