Tool Routing: Match the Model to the Task (2026)
When an interviewer asks how you’d cut an agent’s cost, or which model to call for a given step, they are testing tool routing: send each request to the cheapest tool that is still capable enough. The lazy answer — route everything to the most powerful model — is exactly the one that blows the budget.
Concrete, verifiable example. In the puzzle below, 7 requests each need a capability tier, and each tool has a capability and a cost. Routing every call to the cheapest capable tool costs 84. Defaulting everything to the most powerful model — the common instinct — costs 196: 57% more (2.3×) for identical results.
That gap is the interview. Routing is an assignment problem — match each request to the cheapest tool that clears its capability bar. A senior answer reaches for a small, cheap model wherever it suffices and reserves the expensive one for the calls that genuinely need it. This guide gives you the model, and a free interactive scored against the exact optimum.
Why this question shows up in AI-engineering interviews
Model spend is the cost line every AI team watches, and routing is the lever — so it gets probed constantly, disguised as:
- “Which model would you use for this step, and why?”
- “This agent costs too much per run — how do you bring it down?”
- “How do you decide between a small model and a frontier model?”
- “How would you route requests across a fleet of tools?”
Each is the same assignment problem: every request has a minimum capability it needs, every tool has a capability and a price, and you want the cheapest valid assignment. Interviewers ask because production agents make millions of these calls, and ‘always use the best model’ is the answer that turns a cheap workload into an expensive one.
The mental model that gets you to optimal
Because requests route independently, the optimum is simple to state and easy to get wrong under pressure:
- Match capability to need. A tool qualifies for a request only if its capability meets the request’s need. Among the qualifying tools, take the cheapest — that is the whole optimum.
- Cheap by default, escalate on demand. Most requests are simple and a small model clears them; reserve the frontier model for the few that actually require it.
- Cost isn’t monotonic in capability. A slightly-more-capable tool can be cheaper than a weaker one (different vendors, different pricing). Compare cost among the capable — don’t assume more capable means more expensive.
- Requests are independent. There is no global coupling between them, which is exactly why ‘cheapest capable, per request’ is provably optimal here.
Put together: for each request, filter to the tools that meet its need, then pick the cheapest. Do that for every request and you have the minimum-cost routing — which is what the puzzle scores.
The trap: “just use the best model”
The default wrong answer is to send everything to the most powerful tool “to be safe.” It is safe and it is expensive: you pay frontier prices for requests a small model would have cleared. In the puzzle, that instinct costs 196 against the optimal 84 — 57% more for the same outcomes. Naming the cheapest-capable rule, and that capability is a floor not a goal, is the senior answer.
Practice it: route the calls for the least cost
Describing the rule is not the same as routing a real workload under a budget. Below is the live Tool Router puzzle from The Arena. Assign each request to a capable tool; The Arena computes the cheapest valid routing and scores yours as a percentage of it. No login required.
What “good” looks like
When you route every call to the cheapest capable tool, you can answer “how would you cut this agent’s cost?” with a concrete policy, not a vibe. Beating the default-to-biggest 196 toward the optimal 84 means you’ve internalised that capability is a floor — and you can defend reserving the expensive model for the calls that need it.
Frequently asked
What is tool routing in AI agents?
Tool (or model) routing is sending each request to the cheapest tool whose capability meets the request’s need, instead of using one model for everything. It is an assignment problem; the optimum is the cheapest capable tool per request. In the worked example it costs 84 versus 196 for routing everything to the biggest model.
Should an AI agent always use the most powerful model?
No. Most requests are simple enough for a smaller, cheaper model, and paying frontier prices for them wastes budget. Route each call to the cheapest model that is still capable enough, and reserve the powerful model for the requests that genuinely need it.
How do you reduce an AI agent's cost?
The biggest lever is routing: classify each request’s difficulty and send it to the cheapest capable model rather than defaulting to the most powerful one. Combine that with caching and pruning unnecessary calls.
How do you choose which model to use for a task?
Define the capability the task requires, filter to the models that meet it, and pick the cheapest of those — comparing price directly, since a more capable model is occasionally cheaper. Capability is a floor to clear, not a target to maximise.
New scored, provably-optimal interview drills, now and then — no spam, unsubscribe anytime.
More interview guides
41 puzzles — RAG tuning, tool routing, agent orchestration, prompt/KV caching, memory compaction — each scored against a provably optimal answer.
Explore the puzzles →