Amer Sinha, Google
As enterprises adopt Large Language Models (LLMs), fine-tuning on proprietary data is essential for performance but introduces significant privacy risks. We present JAX Privacy, the actively maintained open-source library recently used to train Google's VaultGemma, proving that Differential Privacy (DP) works at production scale.
Moving beyond theoretical definitions, this talk focuses on the privacy engineering required to deploy secure LLMs. We define a precise threat model—protecting specific fine-tuning datasets from extraction attacks—and demonstrate a complete workflow using the Gemma model family. We provide a data-driven "before vs. after" analysis, contrasting the memorization risks in standard fine-tuning against the guarantees provided by JAX Privacy. Attendees will gain practical insights into managing computational overhead, optimizing privacy-utility trade-offs, and using our latest tooling to audit and deploy privately tuned models in cloud environments.
Authors: Ryan McKenna, Galen Andrew, Borja Balle, Vadym Doroshenko, Arun Ganesh, Weiwei Kong, Alex Kurakin, Brendan McMahan
Amer Sinha is a Software Engineer at Google Research specializing in privacy-preserving machine learning. His expertise spans several areas of applied ML, including computer vision for autonomous vehicles and video analysis, fairness and robustness in large-scale systems, ad prediction models and large language models.
author = {Amer Sinha},
title = {Private Tuning of {LLMs} in Practice: From {VaultGemma} to Custom {Fine-Tuning}},
year = {2026},
address = {Santa Clara, CA},
publisher = {USENIX Association},
month = jun
}