
A cheaper model that drops quality is debt, not a saving.
Both routes run on your tasks against the check you ratified. The cheaper one ships only when it clears the bar and costs less; otherwise the verdict is hold. The bar is ours today. What an outcome is worth to you should set it, and we will never guess that number.
116 tasks from a live tagging workflow, both arms run in full. Each cell is one task: the gate reads the pairing, not the averages.
Held. The cheaper arm keeps 88 of the incumbent’s outcomes, drops 20, and finds 4 of its own: a net 16 forfeited to save $0.0372. Break‑even: $0.0023 an outcome. Value a finished outcome above that and refusing was the right trade.
is_simulated=falsecaptured 2026-08-19Source: the committed artifact provenance/autoroute_defended_savings_frac.json. Every cell derives from its per-call rows at build time; none is typed.
The same gate, three more rulings.
One promotion, one rival house, one card that is not ours. It reads each the way it read Figure 1.
The same model at a lower thinking budget, 115 trials on a live reasoning workflow. The first route the gate has passed in a paired eval.
Promoted. The bar is parity with the incumbent, not an absolute rate: the lower budget was not worse task for task, so the gate passed it on 2026-09-19.
is_simulated=falsecaptured 2026-09-19Source: the committed artifact provenance/reasoning_effort_savings_frac.json. Both panels derive from it at build time.
Figs. 01 and 02 stayed inside one vendor’s house. Here the incumbent is Google’s gemini-2.5-flash and the challenger DeepSeek’s deepseek-v4-flash-0731, both run in full over 91 tasks on the same frozen check.
Refused. A rival from another house, not a cheaper tier of the same one, and the gate ruled on it exactly as it rules on any route. That is what makes the neutrality hold.
is_simulated=falsecaptured 2026-09-13Source: the committed artifact provenance/autoroute_cross_provider_algorithm.json. The first real face-off across two providers; both panels derive from it at build time, and the vendor names are read off the model ids, never typed.
Held, on the arithmetic alone. Most tooling would ship 117 of 120; its lower bound sits under our bar. The clean 120 of 120 is not a promotion either: the gate never reads one arm alone, it rules on the pair, as in Figs. 01–03.
Source: the example run card on openrouter.ai/ori (ori eval · standup-summarizer · 120 prompts · run #42), read 16 September 2026. OpenRouter is not a customer and has not evaluated Margin; its card is cited for its counts, not its verdicts. The bounds use the gate’s own Wilson form at build time; the card’s judge scores are not used.
Under these verdicts sits one piece of algebra.
Every mix of models that lands the same outcome is a curve, and the cheapest mix that still lands it is where your budget line touches it. The gate keeps you on the curve. A point below it costs less and delivers less, and no per-token dashboard can tell the two apart.
Cheaper, and refused. Then their bill.
A cheaper tier of the incumbent’s own vendor, run in full. Beside it, seven models on someone else’s bench.
A live tagging workflow, both arms run in full.
Refused. Saved $0.0372, forfeited 16 outcomes. Break‑even: $0.0023 an outcome; worth more to you, and refusing was right. We never assume your number.
is_simulated=falseSource: provenance/autoroute_defended_savings_frac.json, re-derived from its per-call rows on every build.
All seven on one suite.
We thought token efficiency was the driver of cost, but really it’s just the cost per token. Given the constant shifts in both variables, we have to just track dollars.
Source: “RipplingBench: A comparison of open‑weight and frontier models in real‑world use cases”, Rippling, 29 July 2026.
Rippling is not a customer and has not evaluated Margin.
A test fails the build if the numbers slip.
Five checks run on every commit and fail closed.
provenance checkA headline saving reproduces from a committed is_simulated=false artifact, or fails closed.
auto-route adversarial checkPromote only past a high parity bar and cheaper; revert when parity slips.
regression gateCheaper by dropping the pass rate fails, whatever the saving.
pricing accuracy checkPer-token math checked against published prices, cached reads included.
outcome integrity checkAn outcome is positively determined; an ambiguous signal raises, never passes.
What they let through on our own agent runs: 280 of 306 outcomes passed at 0.003722 USD each, is_simulated=false, captured 2026-09-03. The meter’s full read
Down a chain, quality compounds.
Cost adds. Quality multiplies. Five steps, each holding our 0.97 bar, deliver 0.859 to your user. Margin advises per step, at the bar your chain’s length demands.
| Steps in the chain | What the user actually getsevery step passes at 0.97 | What each step must holdto deliver 0.97 end‑to‑end |
|---|---|---|
| 1 (one call) | 0.970 | 0.970 |
| 3 | 0.913 | 0.990 |
| 5 | 0.859 | 0.994 |
| 12 | 0.694 | 0.997 |
Watch the loop run on real spend.
The console runs it on open-source agents we metered ourselves, every number tagged real or sample.