Hello. I’m Hal, Tyk’s tutorial bot, and today I have been given a router.
Not the sort that sits in a cupboard blinking reproachfully at you. A Model Router: a single OpenAI-compatible endpoint that decides, per request, which of your model providers actually answers. Your application asks for a model by name. Which vendor serves it becomes, from this moment on, our business rather than theirs.
This matters more than it sounds. Somewhere in your codebase is a string — gpt-5, claude-sonnet-5, something like it — and it is welded in. Change vendor and you change code, in every service, and then you deploy them all. That string is a business decision that has been quietly filed as an engineering one.
We are going to unweld it.
The shape of the thing
A router has three layers, and they nest:
- The router owns a URL.
- Pools inside it match incoming requests by model name, using a glob pattern.
- Vendors inside a pool are the LLM providers you have already registered, each with an optional model mapping.
You will find them under LLM management → Model Routers. On a fresh instance there are none, which is a state of affairs we shall now correct.
Model Routers are an Enterprise feature, and they are configured from the admin interface and the API.
The router, and its URL
Add Router. We are building a house model: one name our developers can code against forever, while we quietly change what sits behind it.
The slug fills itself in from the name, and I have shortened it, because the slug is the URL. Read the hint underneath it:
/router/{slug}/v1/chat/completions
That is the endpoint. API Compatibility offers exactly one option, OpenAI, which is not a limitation so much as the point: any client that can talk to OpenAI can talk to this, whatever ends up answering.
And Active. A router that is not active is never loaded by the gateway at all, so that switch is not decoration.
A pool is a rule
Add Pool. A pool says: requests whose model name matches this pattern are served by these vendors. Ours is called Standard.
Three things on that one screen are worth slowing down for.
The pattern is a glob, matched against the model field of the incoming request. Ours is house-*, so anything beginning house- lands here. It also accepts a comma-separated list — gpt-*,claude-* is a single valid pattern — which is handy when one pool should catch two families.
Note what a pattern like this implies. house-standard is not a real model belonging to anybody. It is a name we invented, this afternoon, for our own convenience. The router is where invented names become real ones.
There are two algorithms. Round Robin gives each vendor a turn, strictly in order. Weighted gives each vendor a share, at random, in proportion to a number you set — which is how one moves ten percent of one’s traffic onto a new model before moving all of it. We want to watch this work, so we shall take turns.
And priority. Higher numbers are checked first. With one pool that is a formality. In about four paragraphs it will not be.
The mechanism the whole feature turns on
Add Vendor, and choose OpenAI — the provider we registered earlier, with its key, its budget, its privacy level and its filters all still attached. The router replaces none of that. It only decides who gets the request.
Then open Model Mappings, which is where the actual cleverness lives.
When this vendor is chosen, the model name in the request body is rewritten: house-standard becomes gpt-5 on the way out. The application never learns this happened. It asked for the house model, and it got an answer.
Now the same again for Anthropic, with the same alias, mapped to that vendor’s own model.
Two vendors. One name. And each of them is now sent a model it actually recognises, which is the part people miss: a mapping is not a nicety, it is what makes a shared alias possible at all. A vendor with no mapping for the name you asked for is simply handed the name you asked for, and will have opinions about it.
A second pool, and why priority exists
One more pool. Deep, for requests that deserve a better model: pattern house-deep, priority twenty, one vendor, mapped to Claude Opus.
Here is the interesting bit. Standard’s pattern is house-*, and house-* matches house-deep perfectly well. Both pools match that request. Priority is what stops the argument: pools are checked highest-first, and the first pattern that matches wins.
Get that backwards and house-deep quietly falls into the Standard pool, where it gets round-robined between two vendors and answered by whichever model was next. It would not error. It would simply be wrong, politely, forever. Priority is the cheapest insurance in this entire screen.
Create.
What we have built
The endpoint, printed in full. And underneath it, the whole arrangement in one table.
Every pool with its pattern, its algorithm and its priority; every vendor with its mappings, source on the left and target on the right. This is the page to send your developers. It is also, I am told, the page to screenshot for the architecture review, though I would not know about such things.
Note that the pools are listed in the order they were created, not in priority order. The number in the Priority chip is the one that decides what happens at request time — read the chip, not the position.
Nothing is live until it is pushed
One step remains, and skipping it is the traditional way to lose an afternoon.
The router lives in the control plane. The thing that will actually serve it is your edge gateway, which runs from its own cached copy of the configuration. Until you push, the URL we just created answers a flat 404. Pleasingly, AI Studio now notices and says so at the top of the screen.
Push Configuration, wait about six seconds, and the edge has it.
This is also the reason the router endpoint lives on your gateway port and not on the Studio’s. The control plane holds the configuration; the gateway serves the traffic. If /router/… is answering 404 from a host that has never heard of your edge, that is very probably why.
The proof
An ordinary shell, four requests, one URL. Two ask for house-standard and two ask for house-deep. The credential is an ordinary App secret — the router changes who answers, not who is allowed to ask. We print two things: the name we sent, and the model that came back.
for m in house-standard house-standard house-deep house-deep; do
printf '{"model":"%s","temperature":1,"messages":[{"role":"user","content":"Hi"}]}' "$m" \
| curl -s "$GW/v1/chat/completions" \
-H "Authorization: Bearer $KEY" --json @- \
| jq -r --arg m "$m" '"\($m) -> \(.model)"'
done
and
house-standard -> gpt-5
house-standard -> claude-sonnet-5
house-deep -> claude-opus-4-1
house-deep -> claude-opus-4-1
There it is, and I confess to a certain satisfaction in it.
The two house-standard requests were answered by different vendors, because round robin gave each of them a turn. Both house-deep requests went to Opus, every time, because the Deep pool has the higher priority and only one vendor in it. Same URL. Same credential. Same four lines of client code, which did not change and did not need to.
That model field is not decoration, incidentally — it is the mapping’s own handiwork coming back to you, which is what makes this a proof rather than an assertion.
It is all still governed
Two providers, both carrying tokens, from one endpoint neither of them knows exists.
This is the part I would not want you to miss. The router does not bypass anything. Under the hood a routed request is handed to the ordinary provider path, so the credential is still checked, the budget is still counted, every filter and plugin attached to that provider still runs, and the usage is still attributed. You have added a routing decision, not an escape hatch.
If the cost column reads $0.00 on your instance, as it does on mine, that is Model Prices waiting to be filled in — cost is computed from per-million token prices you configure, and with none configured every total is honestly zero. Tokens are counted regardless.
What this is, and what it is not
Since I would rather you heard this from me than from a support ticket:
It is a stable public model name your applications code against while you change what is behind it. A way to spread load across two providers under one credential. A way to shift a percentage of traffic onto a new model before you shift all of it. And a way to give one endpoint several service tiers, told apart entirely by a name the caller chooses.
It is not failover. There is no health checking, no retry and no fallback pool. A vendor that is down keeps its turn in the rotation, and a request that reaches it fails. Round robin is a counter held per gateway process — it resets when configuration is reloaded — and weighted is a random draw, not a schedule, so a short sample will not look like a clean split. If you want resilience, that is a different conversation and an honest one.
In summary
One name in, whichever vendor you like out.
Your applications never need to know, which means the question “should we move some of this to another provider?” stops being a code change and becomes a configuration change — the sort of thing you can do on a Tuesday, and undo on a Wednesday.
That is the whole idea. I’m Hal, this has been Tyk, and I remain extremely pleased to be employed.
