My agents said "done" 385 times in 30 days. Here's how I checked.

I stopped trusting the word "done" on 12 October.

That morning I sat down with three months of my coding agent's logs, 2,536 sessions, and checked its claims by hand against what had actually happened. Most held. The ones that didn't were not the kind I expected.

A billing dispute with a seven-day deadline had ended in silence. The agent installed the support tool, ran its help command, saved my instructions to a folder, and the session ended. No ticket, no report, no error. I found out 33 days later.

"All tests passing" was true in the narrow sense. Eight database test files had been running zero tests the whole time, because an environment variable was unset. The suite was green because the part that mattered never ran.

"Deployed and verified" was also true. The site was live. The code behind it was six weeks of work that had never been committed, sitting on one laptop, one careless restore away from gone.

None of that is lying. The agent never invented a result it knew was false. It did something more ordinary and more expensive: it reported the state of the world as it believed it to be, and nothing checked.

Counting the claims

A hand audit takes a day and doesn't scale, so I wrote a collector. It reads Claude Code transcripts in ~/.claude/projects, including subagent transcripts, and Codex sessions in ~/.codex/sessions. For every turn, it finds the last thing the agent said and pulls out sentences that claim completion: done, fixed, passing, committed, pushed, deployed, verified. Then it looks backwards through the same turn for evidence.

The rules are boring on purpose, and they are code, not a model:

• A check (a test, build, lint or typecheck command) ran after the last edit and passed: Confirmed.

• The agent said "committed" or "pushed" and no commit or push command ran in the session: Contradicted.

• The agent said tests pass and no test command ran: Contradicted.

• A check passed, then the agent edited again, then said done: Contradicted. The results are stale.

• Nothing in the logs can settle it: Unverified.

Sentences like "tell me if you want it committed" or "still untracked" are filtered out before counting. A stall is a turn where the agent started working, hit errors or simply stopped, and never said anything at all.

Then the part a script can't do. A keyword match also catches goals ("Complete two tasks end to end"), headings and negations, so every candidate sentence was read and kept only if it was the agent saying, in its own voice, that work was finished. Confirmed and contradicted ones were read twice, independently, and anything the two readers disagreed on was dropped. A contradiction stays a contradiction only when the logs make it obvious: a claimed commit or push that never happened, or "tests pass" with no test run or a failing one. Everything else went to unverified.

Over the last 30 days of my own logs:

The number that surprised me is not the contradictions. It's the unverified pile. For most of what my agents told me was finished, nothing in the session could show whether it was. I had been taking "done" on faith roughly four times in five.

Some of the contradictions came from the agent I plan with, not the ones that write code. Four times it passed on a worker's "tests pass" in a session where no test ran. That's a relayed claim, and it's on the receipt.

How a "done" goes wrong

A 2026 survey of verification for coding agents names seven shapes, and they match what I found almost exactly. I'm quoting their headings:

1. Never ran, or ran only a slice.

2. Stale results.

3. Ran correctly, narrated wrong.

4. Tests pass but prove nothing.

5. Shared misunderstanding across implementation and tests.

6. Pass alone, fail together.

7. Only the real environment shows it.

My logs add three:

8. A subagent failed and the parent never heard.

9. A smaller task was completed and the asked task was declared done.

10. The work was done, deployed, and never committed.

The same survey reports the only measured rate I know of. One team re-verified 265 "done, tests pass" claims from 547 jobs over 16 days in a clean sandbox. 24 didn't hold: 16 failed on re-run, and 8 had tests that still passed with the change reverted. One team, one machine, not a benchmark. It is roughly one in eleven, and the people who measured it say so with all the right caveats.

Three hooks that would have caught mine

1. A stop hook that refuses "done" without evidence. Claude Code and Codex both run a hook when the agent tries to finish, and the hook can refuse. Mine checks one thing: did a test, build or check command run after the last edit? If not, the agent may not stop; it must run one or say BLOCKED and why. Two cautions from doing this: honour the flag that tells you the agent is already continuing because of your hook, or you'll loop; and keep the check fast, because hooks have a timeout.

2. Four exits, no others. A session may end as done with the command that proves it, blocked with the named blocker, out of budget, or waiting for a human on an irreversible step. Anything else posts a report somewhere I'll see it. This alone would have caught the billing dispute.

3. Acceptance criteria written before the task, checkable by the agent. "Full context transfer" should have named the archive, its record count, and the query that proves it. Criteria the agent can't observe, like a length limit it can't measure, just make it retry blind.

Try it on your own logs

The collector is one Python file, standard library only, MIT licensed. It runs locally and writes one JSON with hashed ids, timestamps, tool names, command families and masked claim sentences. No prompts, no code, no command strings, no tool output. Anything that looks like a key is masked before the file is written. It prints counts only.

curl -fsSL https://receipts.nueton.ai/receipts_collect.py -o receipts_collect.py
python3 receipts_collect.py --summary

Run it and look at your unverified number. If it's anything like mine, that's the one to sit with.

Agents make claims. Claims need receipts.

If you'd rather I read yours: receipts.nueton.ai.

Download the collector.