Surfacing Hidden Privacy Risks in Code: Lessons from LLM and Retrieval Assisted Detection

Many privacy failures are caused less by obvious code that handles privacy such as encryption, retention, deletion, and more by everyday design and coding choices that unintentionally expose sensitive data. A common example is Default Grant Access (DGA), whereby access is permitted unless a developer explicitly blocks it. DGA is difficult for traditional static analysis because the risk often emerges from context: defaults, conditional logic, and framework behavior.

We report lessons learned from deploying language model-based detection for DGA in pull requests at scale, and from replacing a static prompt approach with retrieval-augmented generation (RAG) to address high false-positive rates. In an evaluation spanning nine production repositories, four languages, and 183,000+ methods, the RAG system identified more than twice as many confirmed privacy-relevant issues as the standalone LLM approach. However, false-positive rates did not improve significantly, and we observed language-specific noise, especially in TypeScript and C++.

We cover practical engineering insights: Curating high-quality examples from known incidents, managing corpus retrieval trade-offs and integrating detections into pull request review workflows without overwhelming reviewers.