Margin and Orchestra: one sells the model it judges, one sells no model.

Orchestra builds evals from your work, trains a cheaper specialist and serves it through its gateway. The difference that matters: the party that sells you the cheaper model should not be the one deciding it passed. Margin sells no model, holds a stated bar, and takes a switch back when quality slips.

Last verified 2026-11-02. Public pages only, read that day. A page we did not find something on is a search result, not proof it does not exist.

Row by row

Who sells the model

Margin's claim

Orchestra

Builds the evals and builds a specialist model where needed; its privacy notice lists Fireworks AI for model hosting, inference and fine-tuning when that workflow is enabled. Calls to Anthropic and OpenAI run on your own provider accounts.

Margin

Sells no model and hosts no inference. Whichever vendor wins on your evals, our flat fee is the same, so we have no stake in which model you run, theirs included.

Sources: Orchestra home, Orchestra privacy notice (updated 2026-09-30), Orchestra terms (updated 2026-05-22), Margin pricing. Read 2026-11-02.

Where it sits

Different designs

Orchestra

A gateway in the request path: you point the OpenAI or Anthropic SDK at its base URL, api.understudylabs.com, with an Orchestra key.

Margin

Not in the request path. A meter beside your calls, so an outage on our side cannot stop your agents. A change arrives as a pull request you merge.

Sources: Orchestra platform, Margin security. Read 2026-11-02.

Getting started

Different designs

Orchestra

Change your SDK's base URL to its gateway and add an Orchestra key, so every production call goes through it.

Margin

Connect a repository or drop a folder. On OpenRouter, turn on Broadcast and paste one endpoint and one header. No code change either way, and no production call ever routes through Margin.

Sources: Orchestra platform, Margin: start, Margin docs: OpenRouter Broadcast. Read 2026-11-02.

The promotion bar

Margin's claim

Orchestra

A new model ships when it meets quality, cost and latency standards, tested against held-out examples. On the pages we read we did not find a numeric bar or the statistic behind one.

Margin

Promoted only when it costs less and the lowest quality your paired evals can vouch for at 95% confidence is at least 0.97 of the incumbent’s. Below that it is held, and the refusal stays on the record.

Sources: Orchestra home, Orchestra platform, Margin: the Parity Gate. Read 2026-11-02.

Held-out confirmation

Both state it

Orchestra

States it: each model is tested against held-out examples.

Margin

A switch picked on some of your cases is confirmed on cases that never chose it, split by a hash of each case, or it is held.

Sources: Orchestra home, Margin: the Parity Gate. Read 2026-11-02.

Size of a change

Margin's claim

Orchestra

Routing rules implement a given model and keep a fallback in place. On the pages we read we did not find whether a change covers one step or a whole route.

Margin

One job at a time. A face-off can move a single step of an agent, every other step as it ran, and the change is a pull request that edits one entry in a model-pin file you own. On our founder's own agents it moved one step this way.

Sources: Orchestra platform, Margin pull request #13 on our founder's own agents (merged 2026-11-02). Read 2026-11-02.

Response time

Both state it

Orchestra

Evaluates quality alongside response time and cost of the complete route.

Margin

Times both arms of a face-off. The pull request prints the median and p90 per model call for each, or says the calls were not timed.

Sources: Orchestra platform, Margin's switch pull request, speed section. Read 2026-11-02.

After the switch

Margin's claim

Orchestra

Ships approved changes with a retained fallback; it can introduce a new route gradually and hold onto the prior route when evidence calls for it. On the pages we read we did not find an automatic revert.

Margin

Watches quality on live traffic after the merge and opens the revert as a pull request when it slips. Undoing a one-job change removes that one entry.

Sources: Orchestra platform, Margin: the Governor. Read 2026-11-02.

Value, not only cost

Margin's claim

Orchestra

The results we read are reported as cost per task, per call or per table, and as speed.

Margin

Price an outcome and Margin holds a switch whose lost passes are worth more than the dollars it saves. With no value supplied we report cost per outcome and say so.

Sources: Orchestra results, Margin: cost per verified outcome. Read 2026-11-02.

When prices or models move

Margin's claim

Orchestra

Not described on the pages we read.

Margin

A daily job re-tests a model when its price drops or its last answer goes stale, and tests a newly released one, with nobody asking. It has run every day since 2026-10-14.

Sources: Margin's production job log (19 daily runs, 2026-10-14 to 2026-11-01). Read 2026-11-02.

Whose model wins

Margin's claim

Orchestra

Trains its own specialist models on your traces, then evaluates them against your current model.

Margin

Builds no model, by design. It picks candidates from the live catalogue of ten labs, and for a one-job face-off you can name any model id your provider serves. It has nothing to gain from which one wins.

Sources: Orchestra home, Orchestra platform. Read 2026-11-02.

Why teams choose Margin

  • A judge with no model to sell, grading every vendor on your own evals.
  • A stated bar: 0.97 of your current quality, at 95% confidence, or no switch.
  • It changes one job at a time, as a pull request you merge.
  • It keeps watching after the merge and opens the revert when quality slips.
  • It re-tests on its own when a price drops or a new model ships.

Choosing between them

Orchestra fits if you want a vendor to build and run a cheaper model for one workflow and you are happy for it to sit in your request path. Margin fits if you want a party with no model to sell to grade every candidate on your own evals, theirs included, and to keep re-testing as prices and models change.

Both test a cheaper model against your evals before a switch. The difference is who does the judging, the bar it holds, how small each change is, and what happens after the merge.

See the gate on real spend.

The case study shows every face-off on our own agent, the refused ones included, read from production.