What this measures: whether yc-software/qm's spend budget — which throttles
agents — computes a number that means what it is used for. Read in source, then
tested against 52,511 real requests.
QM is a multiplayer agent harness, 14,721 stars, TypeScript, pushed the day this was written. It is the sponsor of the Own Your Intelligence hackathon, and this was written to decide whether a pitch built on it would survive contact with the people who wrote it.
Added 2026-09-09, because the first version of this document did not say it and every number below invites the wrong reading.
The corpus is this machine's Claude Code transcripts, and the operator is on a Claude Max subscription. None of these requests were billed per token. The dollar figures are list-price equivalents for someone who is billed that way, computed from provider-reported token counts. Nobody paid $66,177, and nobody paid $8,170 either.
That does not weaken the finding, and it is worth being exact about why. The
8.1x is a ratio of two computations over the same token counts: what QM's
budget.ts charges against what those tokens cost at published rates. Both
sides are arithmetic on the same observations, so the ratio is independent of
who was billed or whether anyone was. The finding would hold identically on a
free tier.
What the billing basis does change is who the harm lands on:
| Reader | What the 8.1x means to them |
|---|---|
| An org billed per token through the API | Money, directly |
| An org running QM with a budget ceiling | Throttling. budget.ts check() returns allowed:false per-principal and org-wide on a 24h window, so a $500/day ceiling stops execution at roughly $62 of real spend |
| A Pro or Max subscriber | Not money. Re-billed tokens still consume the usage allowance, which is a different currency and a separate measurement — see internal/analysis/predictor.go |
replay cost already draws this distinction unprompted, and its wording is the
one to reuse rather than improve on: "On a subscription seat - Claude Pro or
Max, Copilot, Cursor - none of that is money: you are not billed per token, so
the dollars above are list price for someone who is. The tokens are still
yours." The tool was right about this before this document was.
For anyone quoting the 8.1x: it is a defect in QM's budget arithmetic, not a saving Replay produces and not a bill anyone received. Quoting it as "we were overcharged $58,000" would be false.
| Step | Location | What happens |
|---|---|---|
| 1 | harness.ts:140 |
The interface defines cacheUsage { cacheRead, cacheWrite, uncachedInput } |
| 2 | claude-harness.ts:547 |
Anthropic's four usage fields are parsed correctly and separately |
| 3 | claude-harness.ts:570 |
They are collapsed: inputTokens: seen.input + seen.cacheRead + seen.cacheWrite |
| 4 | wiring.ts:1562 |
budget?.record(actorId, estimateCostUsd(rec.inputTokens)) |
| 5 | budget.ts:15 |
(inputTokens / 1_000_000) * usdPerMTok |
| 6 | pi-models.ts:541 |
DEFAULT_AGENT_INPUT_USD_PER_MTOK = 5 |
| 7 | budget.ts |
Per-principal and org-wide, 24h window; check() returns allowed:false at the limit |
Step 3 is the defect. The cache data exists one line above the place it is discarded, and step 7 means the result gates execution for a whole organisation.
Tested against this machine's Claude Code corpus: 52,511 unique requests across three priced models, 63,369 duplicate events dropped by request id.
| Model | Prompt tokens | Cache share | QM charges | True | Ratio |
|---|---|---|---|---|---|
| claude-opus-5 | 13,124,175,407 | 98.7% | $65,621 | $8,036 | 8.2× |
| claude-fable-5-1 | 60,036,134 | 94.0% | $300 | $118 | 2.5× |
| claude-sonnet-5 | 51,214,621 | 95.8% | $256 | $16 | 16.3× |
| Total | $66,177 | $8,170 | 8.1× |
Decomposed, the result is sharper than the headline:
| Correction applied | Total | Ratio |
|---|---|---|
| QM today: flat $5, cache-blind | $66,178 | 8.1× |
| + per-model rates, still cache-blind | $66,831 | 8.2× |
| + cache-aware (correct) | $8,170 | 1.0× |
Ignoring the cache is 101.1% of the error. The flat model-blind rate is −1.1%. It very slightly offsets, because this corpus is Opus-heavy and Opus input happens to be exactly $5.
So the honest version of this finding is not "QM's pricing is wrong". A QM engineer defending the flat rate as an acceptable approximation is right, and the finding survives them completely. One field is dropped, and dropping it costs a factor of eight.
Robust to the one free parameter: at cache-write multipliers of 1.0×, 1.25× and 2.0×, the ratio is 8.3×, 8.1× and 7.5×.
recordModelCall takes { model, inputTokens, entryCount }. Every harness fills
inputTokens from something different:
| Harness | What inputTokens holds |
|---|---|
claude-harness.ts:570 |
Measured usage, cache reads and writes folded in |
codex-harness.ts:495 |
usage.inputTokens — Codex nests its cached tokens inside this |
codex-harness.ts:1185 |
state.fallbackInputTokens — a fallback, not a measurement |
opencode-harness.ts:593 |
countTokens(JSON.stringify(request)) — a local tokenizer estimate |
pi-harness.ts:1750 |
composedPromptTokens + estimateHistoryTokens(...) + countTokens(...) |
Two of those five never consult provider-reported usage at all. They count tokens locally, against a serialized request object, and the result is added to the same org budget as Anthropic's measured figures and Codex's differently-nested ones.
This is larger than the cache. A budget that sums a measurement, a nested measurement, a fallback and two local estimates, then throttles an organisation on the total, is not measuring spend. It is adding up five things that share a variable name.
The distortion is not uniform, and its shape punishes the behaviour the tool should reward.
Cache share on this corpus runs 94–99%. The higher a team's cache hit rate, the larger the fraction of its prompt that QM bills at full uncached price, so the better a team's cache discipline, the harder it is throttled. Sonnet users are billed 16.3× and Fable users 2.5×, on identical behaviour, because the flat rate is 2.5× Sonnet's true input price and half of Fable's.
A throttle wrong by 8× in aggregate and by 6.5× between models does not limit spend. It allocates capacity by a number unrelated to spend.
The inputs already exist. harness.ts:140 defines the three cache classes, and
claude-harness.ts:547 already parses them. The change is to stop collapsing them
at line 570, carry the classes into estimateCostUsd, and price per model.
The harder half is Finding 2, which is not a pricing bug and cannot be fixed by a
price table: opencode and pi do not have provider usage to price. The honest
fix there is to make the budget say which of its inputs are measured and which are
estimated, rather than summing them silently — the same distinction this
repository enforces on itself everywhere else.
Source read at 14,721 stars, pushed 2026-09-08 16:15 UTC. Costs computed against Replay's price table dated 2026-09-07, itself cross-checked against OpenRouter and LiteLLM in TOKEN-PRICES.md. Anthropic multipliers: cache read 0.10×, cache write 1.25× of base input.
The ratios are the result. The absolute totals are not. This extraction reads
$8,170 where replay cost reports $3,244.67 on the same corpus, because the two
scope and deduplicate differently — 52,511 requests here against Replay's 34,184.
Both figures in each ratio come from the same extraction, so any over-counting
cancels; per-model ratios cancel it entirely. No absolute dollar figure from this
document should be quoted as this machine's spend.
One machine, one operator, one corpus, Anthropic models only. The 8.1×
aggregate is a property of a 94–99% cache share. A workload that never caches
would show a ratio near 1, and the finding would be invisible. What generalises is
the mechanism at claude-harness.ts:570, not this number.