Cloak, Honey, Trap: Proactive Defenses Against LLM Agents

Recent advances in large language models (LLMs) have enabled autonomous penetration testing tools capable of assessing network security by compromising hosts. However, the same artificial intelligence (AI) capabilities can empower attackers to automate attacks at scale.

This paper presents a cost-effective defense framework using deception and counterattacks to exploit LLM weaknesses—such as biases, memory limitations, and tokenization issues—to disrupt, detect, or neutralize malicious agents. For example, we are able to cloak assets with misdirection, lure, and expose AI adversaries by using LLM-specific honey-tokens and trap agents using loops and other techniques. We also demonstrate several novel exploits such as inducing an agent to execute untrusted code, potentially giving defenders reverse access to the attacker's infrastructure. Overall, our approach introduces 6 strategies and 15 techniques, most of which do not rely on prompt injection.

With black box assumptions, we are able to protect a variety of 11 different Capture the Flag (CTF) machines with a 100% success rate. To help the community, we release CHeaT, an open-source tool that automatically inserts traps, cloaks, and honey-tokens seamlessly into network assets. This work establishes a scalable proactive defense paradigm leveraging LLM vulnerabilities to counter AI-driven threats.