Cross-provider · never in your path · sells no tokens
Your AI workforce has a payroll. Now price the outcome.
Cost tools say what your agents spent, never what it was worth. Tell Margin what one outcome is worth to you. It reports your return and proves every fix before it ships. Supply no number and you read cost‑per‑outcome, labelled as one.
Sections
- The payrollWhat does one workflow actually cost?
- The evidenceWould you tell me to keep spending, and is that only your opinion?
- The floor testsWhat keeps the numbers honest?
- Our own estateDo you run this on yourselves?
- The loopHow does it decide?
- The chainDoes the 0.97 bar hold down a whole chain?
- What we seeDo you read my prompts?
- 64% cheaperpromoteda lower thinking budget, same model · paired eval, not live traffic · parity 1.06 over the 0.97 floor · 115 tasks
- 69% cheaperrefuseda cheaper tier of the same house · quality fell 13.8 pts · 116 tasks
- 85% cheaperrefusedDeepSeek against the Google incumbent · quality fell 29.7 pts · 91 tasks
The payroll
Every step draws a wage.
A sample chain, labelled simulated, until the live read lands. Agents bill like staff: by the step, whether or not the step was worth staffing. The lit step, draft, takes 45% of the pay, and the steps sum to $0.0344 an outcome: the wage, in one number. To validate it, run the cheaper arm on the workflow’s own tasks and watch. Figure 1 is a cut the gate refused.
Our own estate
The whole estate, read live.
Small, and real: our own work, read when the page loads. A proven saving joins it the day one holds — never before.
| Project | Metered calls | Outcomes | Last seen |
|---|---|---|---|
What is an outcome worth to you?
Only you can price the result: a triaged document, a merged PR, a resolved ticket. Type your number and the chain re‑prices as value per outcome, every outcome counted once and unweighted. It stays in this browser; nothing is sent.
The first plug
Your AI in. Three lines.
Point the meter at the SDK your agents already call; the key is an environment variable and nothing else moves.
// Python: pip install margin-meter
$ npm install margin-meter
$ export MARGIN_INGEST_URL=https://trymargin.io
$ export MARGIN_INGEST_KEY=mgk_…
import { MarginMeter, instrumentGemini }
from "margin-meter";
const meter = new MarginMeter();
instrumentGemini(model, meter, {
workflowId: "support-triage",
}); // every call is now meteredThe other plug
Your evals are the denominator.
Pass or fail is the denominator, and you already grade it: an adapter reads the results file your harness writes and records one outcome per graded case, method stated.
No value supplied — so what you are reading above is cost per outcome, and we say so. No number in mind? Our own gate’s break‑even is $0.0023 an outcome: value a finished outcome above that and its refusal was the right trade, no input needed.
$ promptfoo eval -o results.json
$ margin-meter promptfoo --workflow support-triage results.json
# one outcome per graded case; a skipped case is never a verdict- promptfooPython · TypeScript
- BraintrustPython · TypeScript
- LangSmithPython · TypeScript
- RagasPython
- deepevalPython
- LangfusePython
- pytestPython
- vitest / jestTypeScript
margin-meter SDK with a test beside it in the repo. No customer has run one yet.Cheaper, and refused. Then their bill.
A cheaper tier of the incumbent’s own vendor, run in full. Beside it, seven models on someone else’s bench.
A live tagging workflow, both arms run in full.
Refused. Saved $0.0372, forfeited 16 outcomes. Break‑even: $0.0023 an outcome. We never assume your number.
is_simulated=falseSource: provenance/autoroute_defended_savings_frac.json, re-derived from its per-call rows on every build.
Cheapest and dearest of seven models, one suite, their data.
We thought token efficiency was the driver of cost, but really it’s just the cost per token. Given the constant shifts in both variables, we have to just track dollars.
Source: “RipplingBench: A comparison of open‑weight and frontier models in real‑world use cases”, Rippling, 29 July 2026.
Rippling is not a customer and has not evaluated Margin.
The floor tests
A test fails the build if the numbers slip.
We hold no compliance badge we haven’t earned. Five checks run on every commit and fail closed.
provenance checkHeadline savings reproduce from a committed is_simulated=false artifact; a missing or simulated one fails closed.
auto-route adversarial checkPromote only past a high parity bar and cheaper; auto-revert the moment parity slips.
regression gateA build that cut cost by dropping its pass rate fails, however large the saving.
pricing accuracy checkPer-token math checked against published pricing, cached reads at their real fraction.
outcome integrity checkAn outcome must be positively determined; an ambiguous or empty signal raises, never passes.
What they let through, on the open-source agent runs we metered ourselves: 332 calls, 280 of 306 outcomes passed, cost per outcome 0.003722 USD, is_simulated=false, captured 2026-09-03. The meter’s full read
The loop
One control loop, five parts. 1:41.
Narrated. Nothing plays until you press play.
- Model routingmodel tier
- Reasoning effort / thinking budgetreasoning-effort / thinking-token budget
- Output-length controlmax_output_tokens / conciseness
- Prompt / prefix cachingprompt-cache / prefix-cache configuration
- Batch / flex modebatch API / flex tier for non-urgent work
23 levers in the catalogue · 5 named here
The Money Map
The bill, decomposed to the step that spent it.
Downstream agents keep billing on runs an upstream step already doomed; only a map with outcomes on it sees that.
The Parity Gate
Know where a cheaper model holds.
Both arms run on your own tasks, and it points the other way as often as not: we only tell you to keep spending after running the cheaper option and watching it lose.
Promoted. The lower budget cleared the drawn rule, and the paired test over the same 115 tasks says it is not worse task for task: the first route the gate has passed in a paired eval, on 2026-09-19.
is_simulated=falseThe Governor
Promote, hold, revert. The loop that acts.
A dashboard stops at “you could save this.” A promoted route keeps rescoring on live traffic and auto‑reverts the moment quality slips, with your sign‑off.
Try the gate’s rule. A sandbox — it models the rule, not a recording of live traffic. Parity floor 0.97, the same constant our floor test pins.
Hold Cheaper, and refused. Under the floor, the saving buys lost quality — the incumbent keeps the route.
Your drags never leave the browser, and this verdict is the model’s. The measured ones are the records on this page and on the console.
Down a chain, quality compounds.
Cost adds. Quality multiplies. Five steps, each holding our 0.97 bar, deliver 0.859 to your user, and nothing in your stack computes that: a router decides inside one call, a dashboard totals above it. Margin advises per step, at the bar your chain’s length demands.
| Steps in the chain | What the user actually getsevery step passes at 0.97 | What each step must holdto deliver 0.97 end‑to‑end |
|---|---|---|
| 1 (one call) | 0.970 | 0.970 |
| 3 | 0.913 | 0.990 |
| 5 | 0.859 | 0.994 |
| 12 | 0.694 | 0.997 |
Margin never sees your live prompts.
A meter, not a proxy: we cannot read a request, and an ingest outage cannot stop your agents. The exception is a console run of your own evals on our server.
margin-meter is public on PyPI and npm. Read it before you install it: the privacy policy and DPA list every field it records.
The published package trails this build: npm margin-meter is 0.2.0 (repo 0.3.0; meters calls, no eval adapters). That is what a plain npm install gets. Setup through /start works on it. Registries read 2026-09-23.
On the wire
- workflow name
- provider
- model
- input tokens
- output tokens
- latency
- pass or fail
- computed cost
Plus a link you choose to attach, or nothing.
Never on the wire
- prompt text
- model output
- tool arguments
- your users' data
You do not fire your best engineer to cut salary.
You validate the wage. That is what we do.
Or skip the call: send one workflow’s traces in any form — a Langfuse or LangSmith export, a CSV — and a ranked audit comes back within a day, the runs behind every line, not priced where we could not price it. Send the export.
Or write to subh@trymargin.io. The founder replies, not a sales rep, usually the same day.