The Abuse Uncertainty Principle, and Other Lessons Learned from Measuring Abuse on the Internet

Wednesday, January 29, 2020 - 3:30 pm4:00 pm

David Freeman, Facebook


Fighting spam, phishing, and other forms of abuse on the internet is typically seen as a detection problem: find signals that will identify the bad guys and then use these signals to block them. In this talk, I argue that the most difficult part of fighting abuse is not detecting and blocking the bad guys—it's figuring out whether they're there in the first place. What's the "background level" of spam and fake accounts? How can we figure out what our detection systems are missing? Which abuse problem is the most important one to work on right now?

In this talk, I will show how good measurement of abuse unlocks both prioritization of work and analysis of impact. I will present several approaches that Facebook's integrity teams have used to measure and prioritize their problems, including user reports, human labeling, and automated labeling, and offer scenarios in which each of these should and shouldn't be used.

I will also introduce the "Abuse Uncertainty Principle" which says that you can use a metric for measurement or detection, but not both. The Uncertainty Principle implies that measurement is never a finished project, but I will offer strategies for ensuring that your metrics are good enough to inform key decisions. Armed with these tools, you can go back to your product and find out how much abuse it's attracting, how good you are at stopping it, and where you need to invest next.

David Freeman, Facebook

David Freeman is a research scientist/engineer at Facebook working on integrity problems, with a particular focus on fake engagement, scraping, and automation detection. He previously led anti-abuse engineering and data science teams at LinkedIn. He is an author, presenter, and organizer at international conferences on machine learning and security, such as Enigma, NDSS, WWW, and AISec, and has written (with Clarence Chio) a book on Machine Learning and Security published by O'Reilly. He holds a Ph.D. in mathematics from UC Berkeley and did postdoctoral research in cryptography and security at CWI and Stanford University.

@conference {244746,
author = {David Freeman},
title = {The Abuse Uncertainty Principle, and Other Lessons Learned from Measuring Abuse on the Internet},
year = {2020},
address = {San Francisco, CA},
publisher = {{USENIX} Association},
month = jan,