Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoor

Jinyuan Liu, Tsinghua University; Tianshuo Cong, Shandong Key Laboratory of Artificial Intelligence Security, Shandong University; Pei Li, Shandong University; Tianrui Wang, Tsinghua University; Xinlei He, Wuhan University; Anyu Wang, State Key Laboratory of Cryptography and Digital Economy Security, Tsinghua University; Zhongguancun Laboratory; Xiaoyun Wang, State Key Laboratory of Cryptography and Digital Economy Security, Tsinghua University; Zhongguancun Laboratory; Shandong Institute of Blockchain; National Financial Cryptography Research Center

Although semantic watermarking is considered a promising safeguard for images generated by Latent Diffusion Models (LDMs), the reliance of the watermark detection pipeline on neural networks introduces a critical yet underexplored backdoor attack surface. To systematically study this vulnerability, we propose GhostVAE to plant a stealthy backdoor into the encoder of Variational Autoencoder (VAE), enabling reliable evasion of watermark detection. GhostVAE operates in two stages: it first constructs a universal trigger via power spectrum regularization to improve the trigger robustness, and then trains a backdoored VAE encoder with a parameter-aligned objective. Through extensive evaluations across three state-of-the-art semantic watermarking schemes and three widely adopted LDMs, we show that GhostVAE preserves watermark detection performance on benign images (achieving an average true positive rate of 94.4%), while simultaneously enabling highly effective evasion under trigger activation (achieving an average attack success rate of 94.6%). Moreover, we comprehensively analyze seventeen representative defenses and demonstrate that GhostVAE remains stealthy across the input space, parameter space, and latent space. Our work fundamentally undermines the trustworthiness of semantic watermarking systems and highlights that secure deployment of semantic watermarks requires end-to-end security considerations, particularly for neural network components.

USENIX Security '26 Open Access Sponsored by
King Abdullah University of Science and Technology (KAUST)

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {320809,
author = {Jinyuan Liu and Tianshuo Cong and Pei Li and Tianrui Wang and Xinlei He and Anyu Wang and Xiaoyun Wang},
title = {Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoor},
booktitle = {35th USENIX Security Symposium (USENIX Security 26)},
year = {2026},
isbn = {978-1-939133-58-8},
address = {Baltimore, MD},
pages = {281--300},
url = {https://www.usenix.org/conference/usenixsecurity26/presentation/liu-jinyuan},
publisher = {USENIX Association},
month = aug
}