Joshua Saxe, Abundant Security
Agent development for cyber defense has drifted from science toward craft at exactly the moment agents have begun to do real, complex work where precision matters. The discipline that gave us malware classifiers we could trust, where a validation set told us whether a change helped, does not transfer to an agent that spends an hour per trajectory investigating an exposure across the attack surface of a large enterprise. In this talk I distill lessons from building and deploying agents at scale at Meta and in a startup, and argue for rebuilding our foundations around two transitions. The first is from evaluation in the lab to measurement in production: we must hill climb on the behavior of agents in the world. The second is from improvisation to design patterns: principles for balancing between deciding agent logic ourselves and growing agents through optimization.
Joshua Saxe is CTO and co-founder of Abundant Security, a startup building autonomous agents for cyber defense. Before founding Abundant, he helped start and lead Meta's work on AI for security, security for AI, and LLM security post-training, where teams he led built the foundations of AI security across one of the largest production environments in the world. Earlier, as Chief Scientist at Sophos, he led the research group that shipped neural network malware detection to hundreds of millions of endpoints, work that began with DARPA-funded research at Invincea. He wrote Malware Data Science (No Starch Press), a widely used introduction to applying machine learning to security problems, and has published dozens of papers on AI and security.
