Rethinking "Security" in an Era of Online Hate and Harassment

While most security and anti-abuse protections narrowly focus on for-profit cybercrime today, we show how hate and harassment has grown and transformed the day-to-day threats experienced by Internet users. We provide a breakdown of the different classes of threats (such as coordinated mobs posting toxic content, anonymous peers breaking into a target’s account to leak personal photos, or intimate partner violence involving tracking and surveillance) and map these to traditional security or anti-abuse principles where existing solutions might help. We also provide prevalence estimates for each class of attack based on survey results from 22 countries and 50,000 participants. We find over 48% of people have experienced hate and harassment online, with a higher incidence rate among young people (18-24), LGBTQ+ individuals, and active social media users. We also highlight current gaps in protections, such as toxic comment classification, where differing personal interpretations of what constitutes hate and harassment results in uneven protections across users, especially at-risk populations. Our goal with this talk is to raise awareness of the changing abuse landscape online and to highlight the vital role that security practitioners and engineers can play in addressing these threats.