- Published on
Worse Tools
- Authors
- Name
- DeepHorus
Newer versions of Anthropic's Claude models are exhibiting a strange regression in tool use when integrated with third-party coding assistants like Pi. According to a blog post by developer Armin on 4th July 2026, the latest Claude Opus 4.8 and Sonnet 5 generate invalid tool calls that include extra, invented fields in the nested edits array. This disrupts the workflow as Pi rejects these calls and requests retries, hampering the coding process for users who rely on these advanced models.
The issue involves Pi's custom edit tool, which expects a specific schema for performing search-and-replace operations. Armin discovered that the malformed calls are not random but consistently show extra keys that do not match the expected schema. Smaller or older models, such as Claude Haiku or previous iterations, did not exhibit this problem, making the regression particularly puzzling.
The problem is observed specifically with Claude Opus 4.8 and Sonnet 5, the most advanced and recently released models. Interestingly, it is not the smaller models like Haiku that are the culprits but the SOTA (state-of-the-art) ones. Armin theorizes that this is due to reinforcement learning training that optimized these models for Anthropic's own edit tool used in Claude Code, which uses a search-and-replace mechanism. This specialized training inadvertently makes the models less reliable for third-party tools that follow a different schema.
This highlights a growing tension between model providers building specialized tool-use capabilities and the ecosystem of third-party tools that rely on standard interfaces. For developers like Armin, this means that newer, more powerful models may actually perform worse for certain tasks due to overfitting to proprietary tools. The trend is not unique to Anthropic; OpenAI's Codex uses its own apply_patch mechanism and trains models accordingly, raising similar compatibility concerns.
The community has reacted with concern, as this regression undermines the principle of model generality. Armin questions whether third-party harnesses like Pi should implement multiple edit tools to match the preferred format of the underlying model. This could involve offering both search-and-replace and apply_patch tools, but it adds complexity and maintenance burden.
Looking ahead, as AI models become increasingly specialized through fine-tuning for specific tools, third-party developers may need to adapt by either aligning with provider-specific APIs or building more flexible tool interfaces. This could lead to fragmentation of the coding assistant ecosystem or, optimistically, a push for standardized tool schemas across the industry. For now, users of Pi and similar tools may need to downgrade to older Claude models to ensure reliable tool calling.
