Priciest AI model for every task? How to route by difficulty

An online shop uses 1 premium AI model for every message, from "is it in stock?" to refund complaints. The bill keeps rising, yet most messages need only a short reply. This is an illustrative scenario, not a real measurement.

This article draws on Anthropic's documentation, its post "Building effective AI agents" and 1 arXiv paper, checked on November 4, 2026. All model figures are for Claude; OpenAI and Google documentation was not checked.

It asks whether splitting work by difficulty really costs less, which signals decide the split, and how to measure it.

Key points

• Model routing means classifying each request and sending it to a suitable model: easy work to a small model, hard work to a stronger one.
• Output price per million tokens, read November 4, 2026: Fable 5.1 $50, Opus 5.5 $20, Sonnet 5.5 $10, Haiku 5.5 $0.50 (prompts up to 100,000 tokens).
• Compare cost per finished task, not per token. In Anthropic's internal tests, Opus 5.5 at low effort cost $0.12 per solved task, close to Sonnet 5.5 at default ($0.11).
• Anthropic says tuning effort is often a better lever than switching models. Cheap-first with re-runs only fits when you can check results.
• Anthropic's figures are internal and directional. None exist publicly for Vietnamese, so a business needs its own test set.
• Models retire and prices can change, so keep model names in configuration, not hard-coded.

What model routing is and when it pays off

Model routing is a step that classifies each request before choosing which model handles it. A price question goes to a small model; a complex complaint goes to a large one.

In Anthropic's post "Building effective AI agents" (published December 19, 2024, read November 4, 2026), routing fits when task groups are distinct and classification can be done accurately, by a model or a traditional classifier. Its examples use Haiku 4.5 and Sonnet 4.5, neither among the 4 current models.

RouteLLM (arXiv, June 26, 2024) trains a router that chooses between a strong and a weak model and reports cutting cost by more than 2 times in some cases without losing quality.

The abstract does not name the models tested and Click Star read only the abstract, so treat it as a direction, not figures to apply.

Click Star's view: routing pays off at large volume or when a single task group repeats a lot; at a few dozen requests a day, building and maintaining it can cost more than it saves.

The 4 current models: what the vendor says they are for, and the price

Per the "Models overview" page (no date shown, read November 4, 2026), all 4 models have a 1 million token context window. The vendor's descriptions, cheapest first, with price per million tokens:

• Haiku 5.5: high-volume work needing speed, such as classification, extraction and routing; lowest latency. $0.10 input and $0.50 output for prompts up to 100,000 tokens; longer prompts cost $0.50 and $2.50.
• Sonnet 5.5: a mix of speed and intelligence. $2 input, $10 output.
• Opus 5.5: long-running agentic coding and knowledge work. $4 input, $20 output.
• Fable 5.1: demanding reasoning and long-horizon agentic work; slower. $10 input, $50 output.

Prices come from the "Pricing" page (no date shown, read November 4, 2026), on the Claude API. A token is the unit of text a model reads and writes. The Batch API gives 50% off for work that does not need an instant reply; cache reads cost 10% of the input price (5% for Opus 5.5 and Sonnet 5.5, 2.5% for Fable 5.1).

By Click Star's division, Fable 5.1's output price is 100 times Haiku 5.5's for short prompts. Illustrative example, not a measurement: 10,000 messages of 1,000 input and 300 output tokens cost $250 on Fable 5.1 and $2.50 on Haiku 5.5, before thinking, caching and quality.

Price per token is not price per finished task

Anthropic advises comparing cost per completed task, not per token. The figures below come from the "Optimizing for cost and intelligence" page (no date shown, read November 4, 2026), Anthropic's internal measurements.

On a subset of SWE-bench Pro (software engineering tasks; scores not comparable to the public leaderboard), Sonnet 5.5 at default solved 87.6% of tasks at $0.11 per solved task. Opus 5.5 at low solved 87.4% at $0.12, and at medium solved 92.8% at $0.22.

Fable 5.1 at default solved 92.3% (1 run) at $1.19.

Opus 5.5 costs twice Sonnet 5.5 per token, but at low it costs only about 7% more per solved task because it does less work on each task. The 7% is Anthropic's figure; dividing $0.12 by $0.11 gives about 9%, possibly because the displayed costs are rounded.

Small models do not always lose by much. On GPQA Diamond (graduate-level science questions), Haiku 5.5 scored 85% versus 91% for Opus 5.5 and Sonnet 5.5, at about 1/20 of Opus 5.5's cost per correct answer and 1/10 of Sonnet 5.5's.

What a pricier model buys depends on the work. On DeepResearch Bench II (45 tasks), Opus 5.5 scored 2.6 points above Sonnet 5.5 at 1.5 times the cost, Fable 5.1 6.0 points above at 5.8 times; counting all 50 tasks, Anthropic says neither is measurably ahead.

Splitting work by difficulty: Click Star's 3-tier frame

The "Choosing the right model" page (no date shown, read November 4, 2026) gives 2 ways to start. One starts with Haiku 5.5 and upgrades only when needed, which suits simple high-volume work.

The other starts with Opus 5.5 and steps down; if it falls short at xhigh or max, move to Fable 5.1.

The page says most workloads start with Opus 5.5. The 3-tier frame below departs from that default. It is Click Star's frame, not from outside sources, and not yet tested on real data.

• Tier 1, small model (Haiku 5.5): classifying, labeling, extracting fields, answering common questions from existing documents.
• Tier 2, mid-size model (Sonnet 5.5): drafting replies, summarizing, moderate analysis, with a person reviewing first.
• Tier 3, large model (Opus 5.5, Fable 5.1 if needed): multistep reasoning, high-consequence work, work the lower tier failed.

Signals for splitting work (also Click Star's frame), besides the task type:

• Checkable: if a rule or sample answer exists, try the cheap tier first.
• Cost of being wrong: money, refunds or legal commitments go to a higher tier or need a reviewer.
• Length and steps: long inputs and many steps move up a tier.
• Self-reported confidence: a weak signal; Click Star has read no source proving it reliable, so calibrate it with a test set.

Ways to combine models: routing, escalation and an advisor

Routing first. A cheap classification step assigns the task group, then each group goes to its tier. If classification is wrong, the whole path is wrong.

Escalation. Run the cheap tier, check the result, and re-run only failures at a higher setting. Here "setting" means effort, a parameter that controls how many tokens a model spends when responding, including thinking and tool calls (the "Effort" page, no date shown, read November 4, 2026: 5 levels from low to max).

Anthropic measured Opus 5.5 (SWE-bench Pro): low solved 87.4%, so 13% of tasks failed; re-running failures at high solved about 97% at about $0.17 per task, versus 95.3% and $0.29 with everything at high.

Conditions: a trustworthy failure signal (here, the task's tests), and a failure costs the time of 2 runs. The test changed effort on a single model; applying it to 2 models is Click Star's inference.

Test each level in a separate session. The "Prompt caching" page (no date shown, read November 4, 2026) says changing the top-level effort parameter mid-conversation invalidates the cache, the processed prompt reused at a lower price.

The "Effort" page (read November 4, 2026) says the beta per-message effort change keeps the cache, available on the Claude API for all 4 current models, with a beta header.

Advisor. A small model does the work and asks a large one when stuck (the "Advisor tool", beta, no date shown, read November 4, 2026). An advisor can help only when the main model asks.

On GPQA Diamond the advisor was never consulted in 198 questions. On a coding set the main model plus advisor scored 1.7 points higher (near run-to-run noise) at 2.1 times the cost. On Chartography (a chart-reading set) that pair scored 7 points below the main model alone, beyond noise, and the advisor was consulted on 1 of 300 tasks.

Measure before downgrading

The "Choosing the right model" page calls a test set of your own the most important step.

Anthropic's 4-step method on the "Optimizing for cost and intelligence" page:

1. Sample real requests from logs, write outcome checks, and record cost per task.
2. Measure each model tier at several effort levels and plot score against spend.
3. Only if the curve shows a gap, add a multi-model strategy.
4. Run the winner in shadow on a slice of traffic before switching over.

Metrics (Click Star's frame): pass rate, cost per finished task, escalation rate, time of the slowest requests. Include real Vietnamese messages (no diacritics, abbreviations) with personal data masked.

Risks: retiring models, price changes and model-specific traps

Per the "Model deprecations" page (no date shown, read November 4, 2026), Anthropic gives at least 60 days' notice before retiring a public model. For example, on September 30, 2026 it announced Sonnet 4.5 retires on the Claude API on November 30, 2026.

Tentative "not sooner than" dates on Anthropic-operated platforms: Fable 5.1 on September 1, 2027, Opus 5.5 on September 22, 2027, Sonnet 5.5 on September 28, 2027, Haiku 5.5 on October 7, 2027.

For Haiku 5.5, the "What's new in Claude Haiku 5.5" page (no date shown, read November 4, 2026) says thinking is on by default and counts toward max_tokens, so a small limit can stop the reply before any text.

What this means for Vietnamese businesses

Click Star's view. For most small and mid-size businesses, a reasonable first step is 2 tiers: a small model for checkable classification and extraction, a mid-size model for the rest, with a human reviewer.

Enable the top tier only where the test set shows the lower tier is not enough.

Click Star offers AI integration and AI automation, so it has a related interest; this view is technical.

To do: a 4-week plan

Click Star's reference plan, not an industry standard.

Week 1: inventory

• List every place AI is called and the model used.
• Sample real requests from logs and mask personal data.
• Record cost per request as a baseline.

Week 2: build the test set

• Write a way to check results for each task group.
• Label difficulty and cost of being wrong on the sample.
• Run the sample on 2 to 3 models at 2 to 3 effort levels.

Week 3: design the routing

• Pick a tier for each task group from score and cost.
• Set escalation conditions and log the reason for each escalation.
• Put model names in configuration, not hard-coded.

Week 4: shadow run and monitor

• Run in parallel on a traffic slice; customers' results stay unchanged.
• Compare pass rate, cost per task and escalation rate with Week 1.
• Recheck price and retirement pages; rerun tests when models change.

Common mistakes

1. Comparing models by price per token, not cost per finished task.
2. Downgrading a model without a test set of your own.
3. Using the model's self-reported confidence as the only escalation signal.
4. Escalating with no way to check results, so failures go unseen.
5. Changing the top-level effort parameter mid-conversation when a cache is in use.
6. Hard-coding model names and forgetting retirement dates.
7. Treating the vendor's internal figures as a guarantee for Vietnamese data.

What is not known

No public figures exist on how well each Claude model handles Vietnamese; Anthropic's measurements are internal and mostly about coding and research.

It is unclear how closely escalation between 2 models matches the effort test on a single model, and whether switching models mid-conversation can reuse the cache. The 10,000-message example is illustrative; the 3-tier frame and plan are untested.