Noise Floor

India Today reported on 2026-07-10 that this job "can pay up to a million dollars a year" - the same role The Economic Times reported at $200,000 two months earlier, and because that article body would not fetch, whether the million is base pay, total compensation or one outlier package is unresolved. Anthropic posts no such title at all; the phrase appears in a requirements bullet on its posting for Head of Applied AI Architecture, which states $315,000 to $380,000.

The Wall Street Journal, 2026-07-19: "The hottest job in tech isn't very glamorous." You fly to somebody else's office, log into a network you do not control, and make their mess produce something.

Somebody senior at the customer says the agent got worse this week. You updated the CLI on Tuesday, the customer changed a repo, the vendor shipped four releases, and nobody in the building can tell. That argument has been had in public with no instrument in the room: four Hacker News threads on it took 4,151 points between them, and not one supplies a pinned version pair, a repeat count, or a command a stranger could run. A GitHub repository search for claude code benchmark degradation returns two repositories, one star each. The vendor settled the factual half: Anthropic's April 23 postmortem confirms a real degradation with three root causes and says it "took us over a week to discover and confirm the root cause." At least one cause is server-side, which a client-version bisect cannot see.

So this book does not promise to find the regression you are angry about, and here is the limit up front. No agent ran anywhere in this book. The machine it was written on had no API key and no budget, so every program that spends money - measure.sh, arms.sh, step.sh, the model-graded judge.py - ships written, syntax-checked and never executed, and every pass rate, delta, floor and dollar figure printed here came out of a seeded coin or a number I typed, labelled a fixture on its own page. Chapter 11 says it in one line: "No pin was installed. No agent ran."

What you build across seventeen chapters is bench. One fixed task suite, shipped whole. Hermetic runs under Claude Code and OpenAI Codex at versions you pin on purpose, into one store whose grain is the version. Mechanical graders, because no model grades another model. A cost writer that will not sum across billing bases. A noise floor for your own suite on your own machine class, carrying the repeat count and interval that earned it. A bisect over adjacent pinned releases. A Regression Packet with one minimal command a stranger runs without asking a question. And bench/dash.py, rendering a self-contained page whose chart draws the floor as a shaded band with both version strings and the p-value inside it.

What is proven is narrower. The contamination arithmetic that decides how many repeats your question needs is exact and offline. The graders, the store, the verdict logic, the redaction pass, the packet and issue generators and the dashboard renderer all run offline here, and every output printed beside them was diffed against a real run. So do its refusals. reports/no-boundary-found.md ships in the repository root, verdict line NO BOUNDARY FOUND, because a rig that can only find regressions is not an instrument.

The first pages list what you cannot have. No root cause. No determinism - there is no temperature=0 on the models this rig is pointed at. No cross-runtime dollar total, because the Codex side meters in credits with no documented per-run readout. Every version, price, flag, URL and thread id traces to an official page with the date it was verified. ./verify.sh re-derives what the book's programs print from its own fixtures, offline, in seconds; it asserts nothing about your machine, and an appendix names the three further tiers it does not do.