agent-routing

Agent Routing — model, effort, and cascade selection

The rule that decides everything

Cost is output tokens × output price. Prices span ~5× across tiers. Token counts span up to 7× within a single tier depending on task shape. The shape therefore decides more than the tier does, and routing on the per-token discount gets the answer backwards.

Measured 2026-08-17, 14 spec-dense Python modules graded by hidden tests, all tiers at equal quality where noted:

Haiku is 5× cheaper per token and cost 30% more per solved task than Opus, because it emitted 6.7× the tokens. Prices: Haiku 4.5 $1/$5, Sonnet 5 $2/$10, Opus 5 $5/$25 per MTok.

Two questions before spawning

1. Is the output short or long? Short = a schema instance, a label, an answer, a small patch. Long = a module, a document, a plan, a review.
2. Is it mechanically checkable, or does it need judgment?

Output length is the discriminator because it is what the verbosity multiplier multiplies. Haiku's premium is invisible on a 200-token JSON object and ruinous on a 700-token module that costs it 13,000 tokens of thinking to produce.

Routing table

Haiku holds the top four rows on merit: 240/240 measured across nested modular arithmetic, 30-hop chains, 25-operation state tracking, trap-laden word math, and 5-constraint generation — at effort: low, some with CoT suppressed (references/calibration-2026-07-15.md). Do not up-tier short checkable work "to be safe"; there is no measured benefit and it costs 3–5×. The burden of proof is on routing up.

Haiku loses the generation rows on cost alone, not capability — it scored 14/14 on the same suite Opus swept.

Effort is model-specific — verify per model before reusing a level

Measured 2026-08-17 via per-message output_tokens_details.thinking_tokens, thinking as a share of output on identical prompts:

low is a near kill-switch on Sonnet and a mild trim on Haiku. Sonnet at low dropped 14/14 → 10/14; Haiku at low shed only ~26% of its tokens. So:

• Tune Sonnet with the prompt, not the effort knob. medium is the working floor; low overshoots into thinking-off.
• Tune Haiku with the prompt too, because the knob barely moves it.
• Effort is set on the agent, not per session — an effort inside a per-session model override is silently ignored. Levels: low, medium, high, xhigh, max. Not every model accepts every level; an invalid pair is rejected at agent-create. The create response echoes the resolved config — if effort returns None, the org's beta header (managed-agents-2026-04-01) doesn't carry the feature and the field was dropped, not rejected.
• Buy depth only for judgment-heavy roles; drop triage and formatting roles to low without touching the expensive role's budget.

The concision lever, and its limit

Adding one instruction — this is routine work; do not deliberate at length, do not enumerate test cases or weigh alternative designs; write it directly — cut output 37% on Sonnet and 27% on Haiku, at no quality cost. It composes with effort. Use it on every long-output generation spawn.

Then stop. Thinking below a model's natural level is load-bearing, and cutting into it buys tokens with correctness:

• An engineered suppression prompt (positive framing, bounded budget, n-shot exemplar) cut Haiku 35% and halved its pass rate, 8/9 → 4/9. Within that arm, passing runs thought 1.9× more than failing runs.
• Sonnet at low (2.9% thinking) fell 14/14 → 10/14.
• Priced per passing result the suppressed arms were more expensive: 22,143 tokens vs 17,126 for the un-engineered prompt.

A targeted checklist ("enumerate the spec's rejection rules first") helps only when it names the actual failure mode: it took one validation-heavy task from 15,220 to 9,634 tokens at equal quality, and took a semantics-heavy task from 3/3 to 0/3. Misnaming the failure mode is worse than not intervening.

Cascade

Precondition, checked first: is the cheap tier actually cheaper per task? The first rung is never free, so a cascade pays only when the cheap tier's measured cost per completed task is below the destination's. Verbosity can erase a price discount outright — Haiku at $0.067/task against Sonnet's $0.031 made haiku → sonnet worse than Sonnet alone regardless of p_fail: the attempt cost 2× the destination's entire job. Compute this before designing the ladder.

Second precondition: no verifier ⇒ no cascade. Route by the table instead; silent cheap-tier errors compound with nothing to catch them.

The shape that worked (measured, 14/14 at 0.41× Opus):

result = sonnet(task, effort=low, concise)          # rung 1: 10/14, $0.0155
if verify(result) fails:
    result = sonnet(task, effort=medium, concise,   # rung 2: fixed 12/12
                    prior=result, failure=test_output)

Rungs can be the same model at different effort — often better than a tier jump, because it keeps rung 1 genuinely cheap.

Carry the prior attempt and the raw failure output into the retry. Informed retry fixed 12/12; a blind re-attempt fixed 9/12 and failed one task identically across all three replicates — a systematic blind spot re-rolling never escapes. The extra input averaged 866 tokens, 5.9% of the retry's cost. Input is 1/5 the price of output, so context is nearly free relative to thinking.

Don't pay a frontier model to write guidance. An Opus diagnosis added zero over raw test output in two independent tests, at ~$0.15/task. The failing test already says what the orchestrator would say.

Verify content, not envelope. Strip fences, preambles, and trailing commentary before checking; hard-fail only on semantic content and log envelope deviations as soft. Two Haiku runs returned 7/7 and 6/6 correct fields while both wrapping output in a markdown fence the prompt forbade — a verifier keying on raw.startswith('{') would have escalated both for zero content error. Spurious escalation is a cascade failure mode, not a safety margin.

Judgment tasks fail in a shape checkers miss. Asked to rebut a stakeholder's "spend is down 66%" off a partial-month extract, Haiku killed the bad conclusion but normalized per calendar day across a 40%-weekend window and missed a model-mix confound — while passing every mechanical check available (word count, prose form, internal arithmetic consistency). The cheap tier fails as right headline, missed confound. This is why judgment rows route up rather than cascade.

Context handoff — routing picks the tier; the prompt carries the context

Subagents inherit nothing: not the conversation, not loaded skills, not the existence of artifacts already on disk. Every index, scan output, artifact path, or tool recipe must be serialized into the prompt (or a file the prompt points at). Otherwise the agent falls back to blind rediscovery and the tier premium is spent on crawling. A Sonnet with no handoff wastes more than a Haiku with a good procedure.

Per spawn: (1) artifact paths + how to query them, (2) tool commands verbatim, interpreter path included — subagents don't know your venv, (3) explicit anti-patterns ("no ls/glob discovery"), (4) an output spec.

Evidence: 2026-07-16, four Sonnet Explore agents launched onto a 2,300-file repo without the handoff opened with ls crawls despite a full tree-sitter symbol index sitting on disk; relaunched with per-agent index slices, the verbatim command, and anti-crawl rules, discovery cost dropped to ~zero.

To convert a judgment-shaped task into a cheap-tier-executable one (explicit procedures, n-shot examples), use the sibling down-skilling skill. This skill decides the routing; that one engineers the prompt.

Shared-prefix caching cuts the fan-out multiplier (unmeasured, conditional). When N subagents share a byte-stable prefix — the fixed handoff, not the per-agent slices — prefix caching can pull that portion toward a read-discount rate where the orchestration surface exposes it. Keep per-agent content at the tail. Verify your surface caches subagent prefixes before relying on it.

Loop discipline

Never blind-loop. Re-applying a prompt to a model's own output is the identity at best — an LLM call already unrolls its reasoning internally — and regression-then-freeze at worst: a re-looped haiku broke its own middle line on iteration 2 and froze on the broken text for every iteration after.

1. Loop only with an out-of-band evaluator — ground truth, mechanical checker, or an up-tier judge scoring every iteration.
2. Select, don't trust the last: final = argmax_r eval(answer_r).
3. Stop on first regression. If eval(r) < eval(r-1), stop; loops froze on degraded output rather than recovering.
4. Loop for diversity, not depth. Vary the angle per iteration; identical re-application converges instantly.
5. "Improve this" with no headroom is the danger zone. It pressures the model to change something; without a selector, that change ships.

Judge rules

• Judge model ≠ worker model; judge at least one tier up. Same-model self-assessment is untested.
• Prefer mechanical checkers wherever a spec can be executed (counts, schemas, tests, regex): free, deterministic, zero judge tokens.
• Judges are for rubric quality, not arithmetic — don't ask a model to verify a sum a Python one-liner can check.

Escalation triggers (route up despite the table)

• The verifier fails twice at the same tier.
• The task requires weighing trade-offs with no checkable ground truth.
• Output ships verbatim to a human without review.
• The subagent must plan its own multi-step tool strategy over many turns.
• The task spans multiple sources that may disagree and must be reconciled.

Observing the fan-out — you can't govern what you can't watch

Stop-on-regression and "verifier failed twice" assume you can see a subagent's work while it runs. By default you can't: the session stream previews only the primary thread, and a subagent's output lands only after its whole turn buffers.

Attach one stream per thread. Read the session stream for the coordinator; on every session.thread_created (carrying session_thread_id and agent_name), attach a watcher to GET /v1/sessions/{id}/threads/{thread_id}/stream with event_deltas.

• Preview is a scratch buffer; the buffered event is the record. Deltas are best-effort and shed under load, so concatenated deltas are a prefix of the final text. Reconcile by a single replace when the buffered agent.message arrives; the SDK's accumulate_managed_agents_event folds start/delta/record into one snapshot. One accumulator per connection. (The same trap appears offline: per-message usage records in transcripts include streaming partials — take the max per message id, or you undercount tokens ~2×.)
• No replay. A stream opened after a request started gets no deltas for it, and reconnects never replay — attach on thread_created or miss the first response.
• Coordination events live on the primary thread — session.thread_created, agent.thread_message_sent, agent.thread_message_received. Child tool calls cross-posted to the primary carry session_thread_id; skip them.
• Terminate cleanly. Watchers exit on session.thread_status_idle; the main loop on session.status_idle — print the stop reason when it isn't end_turn, and break on terminated-status events.

Operational, not calibrated. Source: Anthropic Managed Agents notebook CMA_watch_subagents_live (beta managed-agents-2026-04-01); contract in events and streaming.

Measure before trusting this

Everything above is measured on two batteries: a 300-call deterministic calibration (references/calibration-2026-07-15.md) and a 14-task hidden-test coding suite (2026-08-17, ~190 subagent runs). Re-measure when:

• A model or price revision lands. Both the verbosity multipliers and the cost table above invert on either.
• The task family is off both batteries. No deterministic task has made Haiku fail on correctness yet, so the capability cliff is past what's been probed.
• Output length differs materially from what was measured. The whole cost model keys on token volume; a 10× longer artifact re-opens the tier question.
• You need pass-rate differences of 1–2 tasks. Run-to-run variance swamps them: two runs of the same model on the same 14 tasks produced disjoint failure sets and a 23% token gap. Token deltas are trustworthy; small pass-rate deltas are not.