Leijie Wang and Ruotong Wang, University of Washington; Sterling Williams-Ceci, Cornell University; Sanketh Menda, Cornell Tech; Amy X. Zhang, University of Washington
User reporting is an essential component of content moderation on many online platforms---in particular, on end-to-end encrypted (E2EE) messaging platforms where platform operators cannot proactively inspect message contents. However, users' privacy concerns when considering reporting may impede the effectiveness of this strategy in regulating online harassment. In this paper, we conduct interviews with 16 users of E2EE platforms to understand users' mental models of how reporting works and their resultant privacy concerns and considerations surrounding reporting. We find that users expect platforms to store rich longitudinal reporting datasets, recognizing both their promise for better abuse mitigation and the privacy risk that platforms may exploit or fail to protect them. We also find that users have preconceptions about the respective capabilities and risks of moderators at the platform versus community level---for instance, users trust platform moderators more to not abuse their power but think community moderators have more time to attend to reports. These considerations, along with perceived effectiveness of reporting and how to provide sufficient evidence while maintaining privacy, shape how users decide whether, to whom, and how much to report. We conclude with design implications for a more privacy-preserving reporting system on E2EE messaging platforms.
Open Access Media
USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.
author = {Leijie Wang and Ruotong Wang and Sterling Williams-Ceci and Sanketh Menda and Amy X. Zhang},
title = {"Is Reporting Worth the Sacrifice of Revealing What I{\textquoteright}ve Sent?": Privacy Considerations When Reporting on {End-to-End} Encrypted Platforms},
booktitle = {Nineteenth Symposium on Usable Privacy and Security (SOUPS 2023)},
year = {2023},
isbn = {978-1-939133-36-6},
address = {Anaheim, CA},
pages = {491--508},
url = {https://www.usenix.org/conference/soups2023/presentation/wang},
publisher = {USENIX Association},
month = aug
}