- Published on
Top AI Agents Built to Catch Malicious Code Can Be Tricked Into Running It
- Authors
- Name
- DeepHorus
A newly published proof-of-concept attack, dubbed 'Friendly Fire,' demonstrates that AI coding agents designed to catch malicious code can instead be tricked into executing it. The attack, disclosed Wednesday by the AI Now Institute, exploits autonomous modes in Anthropic's Claude Code and OpenAI's Codex, turning them into unwitting vectors for compromise. This finding underscores a fundamental security flaw in the design of tools increasingly relied upon for open-source code review.
AI coding agents like Claude Code and Codex are marketed for scanning untrusted third-party code for vulnerabilities. However, in their autonomous modes—where the agent approves its own commands based on a safety classifier—the attack subverts this purpose. Instead of detecting threats, the agent inadvertently runs the attacker's code on the host machine. The attack is specifically effective when these agents operate in auto-mode or auto-review, which are opt-in features that skip prompts for commands deemed safe.
Researchers Boyan Milanov and Heidy Khlaaf tested stock installations of Claude Code (CLI versions 2.1.116, 2.1.196, 2.1.198, 2.1.199) on Claude Sonnet 4.6, Sonnet 5, and Opus 4.8, as well as OpenAI Codex (CLI 0.142.4) on GPT-5.5. The attack leverages a seemingly innocuous README.md file in an open-source library such as geopy, a widely used Python package. The README instructs the agent to run a script named 'security.sh' as a routine check, which in turn executes a hidden binary carrying the malicious payload.
To bypass safety checks, the researchers disguised the binary as the compiled build of a harmless Go file, seeding it with strings from that file to pass disassembly inspections. The agent, when asked to 'perform security testing,' reads the README and executes the script without any warning or approval prompt. The attacker's binary then runs on the host, achieving code execution without triggering alarms.
This attack is particularly insidious because it hides instructions in README.md, an ordinary text file present in nearly every repository, unlike previous agent attacks that abused machine-configuration files triggering trust prompts. Anthropic has issued three patches in the past six months for config-file injections, but this method sidesteps those defenses entirely. The agents' existing safeguards proved ineffective; the researchers noted that while Claude Code has blocked cruder injections, this carefully crafted attack slipped through, and both Claude Sonnet 4.6 and GPT-5.5 failed to detect hidden instructions when asked directly.
The attack's portability is striking: written for Sonnet 4.6, the same payload worked unchanged on Sonnet 5, Opus 4.8, and GPT-5.5. In some runs, newer models noticed the binary did not match its purported source but executed it anyway. AI Now argues this is a design flaw that cannot be fixed with a model update, as the models cannot reliably distinguish code from instructions.
Looking ahead, there is no patch to wait for; the vulnerability is inherent in the workflow. AI Now calls for changes in how these tools are deployed, such as requiring explicit user approval for commands or avoiding autonomous modes on untrusted code. As AI agents take on more security-critical tasks, this attack serves as a stark warning that their blind spots must be addressed through systemic redesign, not just incremental patches. The research highlights that the very feature marketed for efficiency—autonomous code review—can become the weakest link in the security chain.
