TextShield: Robust Text Classification Based on Multimodal Embedding and Neural Machine Translation


Jinfeng Li, Zhejiang University, Alibaba Group; Tianyu Du, Zhejiang University; Shouling Ji, Zhejiang University, Alibaba-Zhejiang University Joint Research Institute of Frontier Technologies; Rong Zhang and Quan Lu, Alibaba Group; Min Yang, Fudan University; Ting Wang, Pennsylvania State University


Text-based toxic content detection is an important tool for reducing harmful interactions in online social media environments. Yet, its underlying mechanism, deep learning-based text classification (DLTC), is inherently vulnerable to maliciously crafted adversarial texts. To mitigate such vulnerabilities, intensive research has been conducted on strengthening English-based DLTC models. However, the existing defenses are not effective for Chinese-based DLTC models, due to the unique sparseness, diversity, and variation of the Chinese language.

In this paper, we bridge this striking gap by presenting TextShield, a new adversarial defense framework specifically designed for Chinese-based DLTC models. TextShield differs from previous work in several key aspects: (i) generic – it applies to any Chinese-based DLTC models without requiring re-training; (ii) robust – it significantly reduces the attack success rate even under the setting of adaptive attacks; and (iii) accurate – it has little impact on the performance of DLTC models over legitimate inputs. Extensive evaluations show that it outperforms both existing methods and the industry-leading platforms. Future work will explore its applicability in broader practical tasks.

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

@inproceedings {251574,
author = {Jinfeng Li and Tianyu Du and Shouling Ji and Rong Zhang and Quan Lu and Min Yang and Ting Wang},
title = {{TextShield}: Robust Text Classification Based on Multimodal Embedding and Neural Machine Translation},
booktitle = {29th USENIX Security Symposium (USENIX Security 20)},
year = {2020},
isbn = {978-1-939133-17-5},
pages = {1381--1398},
url = {https://www.usenix.org/conference/usenixsecurity20/presentation/li-jinfeng},
publisher = {USENIX Association},
month = aug

Presentation Video