Where Margin fits among the tools you have.

Routers, spend dashboards, eval tools, model builders and the harness you build each do something real. Figure 1 shows what three of them see on one route we measured.

One cheaper route, read at three scopes. Only the one that holds the outcome saw quality fall, so only it refused the 69% saving.Each row reads the same committed run. The cost bars are identical on purpose: every scope sees the price. What differs is whether quality on the task is in view.Figure 1
ScopeCost, against the incumbentQuality on the taskWhat it concludes
A routerreads one callincumbent 100%candidate 31%Not in its scope. Its own score, where it has one, grades the reply.Cheaper per call: route to it.
A spend dashboardreads the invoiceincumbent 100%candidate 31%Not in its scope.A 69% smaller bill: report a saving.
Marginreads the call and what it landedincumbent 100%candidate 31%incumbent 93%candidate 79%Worse on the task: refused.

incumbent modelcheaper candidate

n = 116 tasks, both arms in fullquality = exact match on the taskcost −69% against the incumbent armis_simulated=falsecaptured 2026-08-19

Source: the committed artifact provenance/autoroute_defended_savings_frac.json, read at build time.

Seven kinds of tool, and where Margin differs

Spend dashboards

What it doesEvery dollar across cloud, SaaS and AI vendors in one place, with budgets and chargeback.

MarginReads that spend, then tests a cheaper model on your evals and opens the change as a pull request. A dashboard reports; it does not switch the model.

Observability and evals

What it doesTraces each call and scores the answers with your evals.

MarginTakes those eval results as the grade, through adapters for the common eval tools, and decides whether a cheaper model still passes them.

Routers and gateways

What it doesOne API to many models. Picks a model per request on price or speed, or fails over when a provider is down.

MarginStays out of the request path, decides which model a workload should run on your evals, and re-tests when prices and models move. A router can carry the switch.

Model builders, such as Orchestra

What it doesBuilds evals from your traces, trains a cheaper specialist and serves it through its own gateway.

MarginSells no model, so it can judge any candidate on your evals, a builder's included. Margin vs Orchestra.

Cost ledgers, such as Revenium

What it doesRecords cost per job beside an outcome and a value the caller supplies, and prices tools, APIs and human review too.

MarginGrades the outcome with your eval instead of taking it as given, and acts on the result. Margin vs Revenium.

A cloud's agent platform

What it doesRuns the agents: runtime, memory, tool gateways and traces. The same company sells the inference.

MarginSells no inference, so the verdict on which model to run does not come from the party billing for it. When it is the right buy.

Your own build

What it doesPer-call metering is a sprint for a strong team.

MarginThe hard part is trusting the measurement enough to act on it, and to undo the action. What a home-built harness gets wrong.

Row by row

● covers you◐ partly, with a caveat○ leaves it to you
Four concerns, three categories. Every row is a fact about a category, not a product.
The concernA routergateway · model pickerA spend dashboardFinOps · cost trackingMarginlabor economics for agents
The invoiceleaves it to you. Sees only the traffic it routes. The rest of the bill is elsewhere.covers you. Every dollar across every vendor, with budgets and chargeback.partly, with a caveat. Reads it. Does not replace it.
What the money boughtleaves it to you. Cost per call. Where it scores quality, the score is not your outcome.leaves it to you. Cost per vendor, per month. A merged ticket and a timeout cost the same.covers you. Cost per outcome: the call and what it landed, side by side.
When quality slipsleaves it to you. Stays on the cheaper model until someone notices.leaves it to you. Reports the saving. The regression is on somebody else's chart.covers you. Opens the revert as a pull request and withdraws the saving when quality slips after the merge.
Who sells you tokenspartly, with a caveat. Many resell inference at a markup, or run inside the platform that bills you for it.covers you. Nobody. It sells the view of the bill, not the inference.covers you. Nobody. You buy from whichever vendor wins on your evals, and our flat fee does not rise with your bill.

When another tool is the right buy

A router

Chooses a model per request. A real job, and one move inside our loop.

The right call if

  • Per-call model selection, or failover when a provider goes down, is all you need.
  • You already trust the smaller model on your own traffic.

A spend dashboard

Puts every dollar across your vendors in one place. We read that data rather than compete with it.

The right call if

  • One picture of every dollar across cloud, SaaS and AI vendors is the goal.
  • Budgets, chargeback and finance-grade reporting matter most.

A cloud's agent platform

Runs the agents: a managed runtime, memory, tool gateways, its own traces and quality signals, with models from inside and outside that cloud. It is also the party selling you the inference.

The right call if

  • You want a managed runtime, identity and tool gateways, and your agents live in one cloud.
  • Traces, quality signals and the platform running the agents belong in one place.

Your own build

Per-call metering is a sprint for any strong team. The section below is about what comes after it.

The right call if

  • One workflow, one model, and a spare engineer describe your situation.
  • Spend is too small to justify a tool, and building appeals more than buying.

Cost per outcome is the honest default of a value-per-outcome model. Until you price an outcome, every figure Margin computes is cost per outcome, and it is labelled as one. Price yours and the meter records it beside the cost, every outcome counted once and unweighted. A value you did not supply is never invented.

You can build the meter. Trusting it enough to act on is the hard part.

On 6 paired tasks our own gate measured parity 1.00 and promoted a cheaper model. The same route on 116 tasks measured 0.85, under our bar, and we withdrew the promotion. 6 easy tasks cannot separate two models, and a harness that stops at the first green never learns it shipped a regression. The take-back is on the record.

Measuring is a weekend. Trusting the measurement enough to act on it, and to undo the action, is the year.

What a home-built harness gets wrong

A panel too small to decide.
Parity on 6 tasks is the absence of evidence, and it reads identical to a pass.
Call sites nobody instrumented.
On a real bill, 3 of 6 sites were unmetered and carried all of the cache-write spend.
A price table that drifts.
One output rate moved 2.4× in a day; a hardcoded table takes every past number with it.
A cold start on every task class.
The one you cannot build: a prior learned from traffic that is not yours.

The five questions buyers ask any AI vendor

Buyers run this script before they sign anything. These are our answers, and where each one shows.

Will a successful engagement end with more of your people on my account, or fewer?
Ours ends at one person, and it starts there too. Margin is software plus its founder: no bench, no hourly billing, and no delivery team that grows with your account. Who builds it.
When does billing start, and what am I paying for in month one?
Billing starts when a pilot starts: $2,000 for 45 days on one agent workflow, credited to the first month if you continue. The pilot ends in a change that passed your quality gates, got merged, and re-measured as cheaper, or in a plain statement that we found none worth making. The terms are on the pricing page.
Show me one workflow you operate every week without you in the room.
Our own project. Its model calls are metered as they happen, the re-measure runs weekly on a schedule with nobody driving it, and the console shows the face-offs it ran with each verdict or refusal, every number tagged real or simulated.
Who owns this the day you leave, and what is the single number we are moving?
You own it. The meter is the open-source margin-meter package, on PyPI and npm, and the rows land in your project. The number is your provider invoice, reconciled against the spend we measured; that reconciliation is the pilot’s pass or fail test.
How much smaller has your own delivery team gotten as the models improved?
It started at one person and it is still one, while the routines and the console carry the recurring work. We count no saving from a team that never existed.

Comparing against one vendor in particular? Read Margin against Orchestra or Margin against Revenium, or why cost per verified outcome is the number we measure.

See the comparison run on real spend.

The console shows the loop on open-source agents we metered ourselves: measured spend, recorded face-offs and the gate’s verdicts, every number tagged real or simulated.

or email subh@trymargin.io directly; it reaches the founder, not a queue.