Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis


Xudong Pan, Mi Zhang, Yifan Yan, Jiaming Zhu, and Min Yang, Fudan University


Among existing privacy attacks on the gradient of neural networks, data reconstruction attack, which reverse engineers the training batch from the gradient, poses a severe threat on the private training data. Despite its empirical success on large architectures and small training batches, unstable reconstruction accuracy is also observed when a smaller architecture or a larger batch is under attack. Due to the weak interpretability of existing learning-based attacks, there is little known on why, when and how data reconstruction attack is feasible.

In our work, we perform the first analytic study on the security boundary of data reconstruction from gradient via a microcosmic view on neural networks with rectified linear units (ReLUs), the most popular activation function in practice. For the first time, we characterize the insecure/secure boundary of data reconstruction attack in terms of the neuron exclusivity state of a training batch, indexed by the number of Exclusively Activated Neurons (ExANs, i.e., a ReLU activated by only one sample in a batch). Intuitively, we show a training batch with more ExANs are more vulnerable to data reconstruction attack and vice versa. On the one hand, we construct a novel deterministic attack algorithm which substantially outperforms previous attacks for reconstructing training batches lying in the insecure boundary of a neural network. Meanwhile, for training batches lying in the secure boundary, we prove the impossibility of unique reconstruction, based on which an exclusivity reduction strategy is devised to enlarge the secure boundary for mitigation purposes.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {279924,
author = {Xudong Pan and Mi Zhang and Yifan Yan and Jiaming Zhu and Zhemin Yang},
title = {Exploring the Security Boundary of Data Reconstruction via Neuron Exclusivity Analysis},
booktitle = {31st USENIX Security Symposium (USENIX Security 22)},
year = {2022},
isbn = {978-1-939133-31-1},
address = {Boston, MA},
pages = {3989--4006},
url = {https://www.usenix.org/conference/usenixsecurity22/presentation/pan-exploring},
publisher = {USENIX Association},
month = aug

Presentation Video