- Published on
ProofRun – a local verification receipt for AI coding agents
- Authors
- Name
- DeepHorus
ProofRun, a new open-source verification tool, launched to give AI coding agents a cryptographic receipt for claims that tests pass. Version 0.2.0 is now available on GitHub, offering developers a way to distinguish between "I ran it and it passed" and "I'm pretty sure it would pass." The tool is designed for a growing problem: AI agents often assert that all tests are green after multiple code edits, but those assertions may be stale or inferred. ProofRun closes this gap by making the claim itself checkable, not by relying on the agent's honesty.
The tool works by running real subprocesses and reading their actual exit codes, binding each result to a fingerprint of the exact code state. That fingerprint includes the git commit plus a hash of everything uncommitted, whether staged or not. Change a single byte, and the result automatically flips to STALE. ProofRun defines four statuses—PASS, FAIL, STALE, and NOT RUN—each derived from observed execution or the documented absence of one. There is no fifth "probably fine" status.
ProofRun v0.2.0 is fully offline, makes zero network calls, sends no telemetry, and requires no accounts. It does not use AI to verify AI; instead, it starts a real subprocess and reads its exit code. The command comparison is argv-exact, not string-matched, so a check declared as pytest -k "foo bar" cannot be satisfied by a visually similar command. The tool does not parse test output, judge code quality, or auto-fix anything.
For development teams, this means a concrete way to audit what an AI coding agent actually did. The proofrun status --strict command exits with a non-zero code if a required check is not PASS, making it easy to integrate into CI pipelines. Developers can see at a glance whether a result is still valid for the current code state, without having to remember to ask if a PASS still counts. This addresses the core trust gap in AI-assisted development.
ProofRun itself was written by an AI coding agent, Claude Code, under human direction, then subjected to several rounds of independent, read-only adversarial review before release. That review uncovered a real flaw: a misquoted shell argument could make a check silently run zero tests and still report PASS. The developers fixed it after reproducing the issue and verifying the remedy against the actual reproduction, not just reviewing for plausibility. The full case study is documented in docs/case-study.md.
Looking ahead, ProofRun is positioned as essential infrastructure for AI-assisted software development. By holding AI agents accountable with the same scrutiny applied to the tool itself, it sets an example for how verification tools should be built and tested. As AI coding agents become more common, tools like ProofRun may become standard practice for ensuring reliability. The project is open source, allowing the community to contribute and extend its capabilities.
