Switch models only when your evals say it holds.

Margin tests cheaper models on your own evals, opens a pull request for the ones that hold, and a revert if quality slips.

SIMULATED sample data below, not a customer

One press of Preview the flip moves each share. Simulated data; dollars blurred.
SIMULATED · sample data
The Margin console on simulated sample data, dollar figures blurred: a top fix with a Preview the flip button, the sample gate deciding one promoted against two refused, and the money map of four sample systems.

Cheaper only where quality holds.

23.2%

less per passed outcome on our own agent, 120 of 120 evals still passing.

Both models passed every case, so the evals could not tell them apart and a person chose.

  1. Beforegpt-4o-mini$9.00per million passed · 120/120 · run #26
  2. After 2 pull requestsministral-8b-2512$6.92per million passed · 120/120 · run #88
Real record · our own agent, not a customer · 2 pull requests a person merged · captured 2026-10-17 from productionEvery run, refusals included

It turns down a saving when quality falls.

On the same agent, a cheaper model failed the evals. Margin refused it and opened no pull request.

More results, each on its own record: the merged switches on our founder’s agents, each re-measured after its merge and the paired evals behind the gate.

REAL RECORD · our own agent
Refused.

67% cheaper per task, and it passed 72 of 120, 0.60 of the incumbent’s quality. No pull request was opened.

Incumbent gpt-4o-mini120 of 120 passed · $9.00 per million tasks run, passed or not
Cheaper candidate mistral-small-3.2-24b-instruct72 of 120 passed · $2.94 per million tasks run, passed or not

The black tick is the gate’s floor: 0.97 of the incumbent’s quality.

Run #27 · read from production 2026-10-17 · not a customerThe full record

The loop, in four steps.

How each step works
  1. MeasureMargin meters every call, and your own evals are the outcome count.
  2. TestIt tests cheaper models on those evals.
  3. SwitchOnly a cheaper model that holds quality ships.
  4. RevertIf quality slips, it goes back to the model that held.

Margin never sees your live prompts.

A meter, not a proxy: we cannot read a request, and an ingest outage cannot stop your agents. Margin reads your repo and your eval results, never sits in your request path, and changes nothing in your code: it opens a pull request you merge, and if quality slips it opens the revert as a second pull request. The exception is a console run of your own evals on our server. What is on the wire.

The right model keeps changing. Margin re-tests on its own.

  • Value first.

    Price an outcome and Margin weighs what a switch saves against the passes it loses. With no price, it reports cost per outcome and says so. The number we measure.

  • It keeps checking.

    When a model's price drops or a new one ships, Margin tests it on your evals without being asked, and asks an old answer again once it goes stale.

  • No model to sell.

    Margin builds, hosts and resells no model. Its flat fee is the same whichever model wins on your evals.

How Margin compares to routers, dashboards and model builders

Margin checks before it swaps, and says no when the work doesn’t hold.

Or paste this into Claude Code, Codex or Cursor:

Follow https://trymargin.io/mcp-prompt.md to connect Margin to this agent.