Anyone can log a token count. The hard part is proving what it bought.
A small SDK wraps your LLM calls and records the outcome the moment your own check fires: a merged PR, a passed test, a resolved ticket. An emit never raises into your request path, a row that fails to land is reported rather than lost silently, and the source is forced server-side to your project.
See the meter running ↓Open-source coding agents (Aider, CrewAI) on gpt-4o, wrapped call by call. Each row below is one call’s real bill; the denominator counts only outcomes the workflow’s own check landed. A call that fails still bills — it just adds nothing to divide by, which is the whole difference between cost per call and cost per outcome.
A measurement, not yet a proof. These are open-source agents on our own harness, not a customer’s estate — the method is real and the population is ours. Every downstream verdict — the Money Map, the Parity Gate’s face-offs — divides by outcomes counted this way.
is_simulated=falsecaptured 2026-09-03Source: the committed artifact provenance/oss_cost_per_outcome_usd.json. Every row and the division derive from it at build time; none is typed.
The meter, still running.
Start with one workflow. Every number Margin reports is a cost divided by an outcome: guess the outcome, and cost per outcome, retry tax and every parity verdict downstream inherit the guess. So the Meter waits for a real definition of “worked” instead of inventing one. The denominator is the whole measurement.
Source: GET /api/economics/summary?simulated=false&public=true. A number that did not arrive is not approximated here.
That definition rarely needs building. If a test suite or an eval harness already grades this workflow, an adapter reads what it writes, and every graded case lands as an outcome, pass or fail, with its grading stated. The whole integration is those two plugs: the meter on your calls, an adapter on your evals, and the second plug is a file your tools were writing anyway.
Because each outcome keeps how it was graded — a code assert, an LLM judge, a person — your console shows the mix, and how many judges stand behind it. That is a record of what graded the denominator, not a score of your evals: we do not grade work we cannot see.
Each one plugs into the tool’s own harness or reads its results file and records one outcome per graded case. A skipped or errored case is never counted as a verdict, and none of them adds a harness of ours to your suite.
- promptfooPython · TypeScript
- BraintrustPython · TypeScript
- LangSmithPython · TypeScript
- RagasPython
- deepevalPython
- LangfusePython
- pytestPython
- vitest / jestTypeScript
Source: the adapter modules in the margin-meter SDK, one test per adapter. The published package trails this build: npm margin-meter is 0.2.0 (repo 0.3.0; meters calls, no eval adapters). That is what a plain npm install gets. Setup through /start works on it. Registries read 2026-09-23.
The one field it cannot fill.
There is one field the Meter has no way to fill: what the outcome was worth to you. It records the cost, the verdict your own check returned, and a quality score when your grader emits one, and not one of those is a figure in dollars. Cost per outcome then divides by the outcomes that passed, which weighs them equally by construction. What one of them is worth is yours to state, and we will not guess it.
Cost per outcome is the honest default of a value-per-outcome model. No customer has priced an outcome yet, so every figure Margin computes is cost per outcome, and it is labelled as one. Price yours and the meter records it beside the cost, every outcome counted once and unweighted. A value you did not supply is never invented.
Source: GET /api/economics/summary?simulated=false&public=true. A number that did not arrive is not approximated here.
- The Meter: measures every call, its cost, tokens, substrate, and whether it worked.
- The Estate: the org chart of your AI workforce.
- The Money Map: where the spend goes, decomposed by step.
- The Parity Gate: proves the cheaper route held quality before it ships.
- The Governor: acts with a human ratifying, and a revert armed; if quality slips it reverts and the loop re-runs.
Watch the loop run on real spend.
The console shows the loop on open-source agents we metered ourselves: measured spend, recorded face-offs and the gate’s verdicts, every number tagged real or sample. No live route has slipped yet, so the revert has not fired. When you have seen it, an email address on /start gets you a project and an ingest key in seconds.