Skip to main content

Routing and failover

Updated on 3 November 2026.

For every request the gateway decides three things, in this order:

  1. Which route applies: a Policy Engine route, an app routing group, or just the requested model.
  2. Which model and provider in that route: kept the same for the whole conversation so the provider's prompt cache keeps hitting.
  3. What to do when that provider fails: retry it, or move the request to a backup.

With the Policy Engine on for the LLM Gateway, the Routing Groups & Fallbacks control owns all three. Without it, the app's Routing tab does; see Routing without the Policy Engine.

Check access
Allowed Models
Requested model
Pick the route
Content routes
Route to
Routing groups
Pick the model
Conversation pin
Weights
Reach it
Native or converted
Send
Retry
Failover
Pin
After a successful response
QuilrAI

Configure routing​

Choose your gateway's mode. Classic app routing settings stay stored while the Policy Engine is on; published policy routes decide live traffic.

Where it lives​

Open Govern > Policy Engine > LLM Gateway > Controls > Routing Groups & Fallbacks. Use Start from a recipe for a common setup, or New route for another route.

Panel tabWhat it shows
RoutesRoutes by rank, routing limits, recipes, and configured destinations.
Failover defaultsRecovery settings for routes without their own failover configuration.
ActivityObserved routing behavior and the rules/settings that decided it.

Use Search routes, models or groups and the All, Limits, Split, Try in order, Content, and Watch only filters to find a route. Configure the scope and destination, check the readable rule summary, then Save to draft. Live traffic changes after Review & publish.

Routing limits apply across matching routes and during failover, regardless of route rank. Without a matching route, requests go to the model the client asks for.

From scripts and pipelines​

Everything on this card can also be managed with the Management API, as typed resources: routes, weighted and ordered destinations, failover profiles, rules and default, routing constraints and model access. Admins never need to write QuilrQL for these: each API write is validated, turned into policy rules and published as one revision, with dry runs, ETags, revision history and restore.

TaskAPI
Model aliases, weighted splits, gradual migrations, failoverRouting and failover API
Request and token limits, request guards, temporary raisesRate limits API
Take a provider out of routing during an outage, reset cooldowns, drop pinsIncidents API

Rules written in the console appear in API lists as read-only rows; API-managed routes and console rules are ordered as described in Rank and precedence.

While the Policy Engine is on, the app's Routing tab is frozen: app routing groups and custom routing are ignored. Converted app groups become policy groups scoped by Requested model; see What happens to classic settings. Provider credentials, endpoints and the enabled switch stay live and are looked up by provider label.

How a request is routed​

Step 1: which route applies​

  1. Does Allowed Models reject the requested model?
    yes:
    403 model_access_denied
  2. Does the request continue state held by one provider?previous_response_id, cachedContent
    yes:
    The credential that holds itNever moved
  3. Has this conversation already been given a destination?Keep destination routes, matched data routes
    yes:
    Same destinationUnless a higher route takes over
  4. Does a content route match?Highest in the list first
    yes:
    Route's destination
  5. Does a Route to or Routing group match?Higher priority wins; a tie goes to Route to
    yes:
    Targets or weighted split
  6. None of the above
    Requested modelOne provider serves it, or one is picked and kept
QuilrAI
  • Rejected models on the Allowed Models card win over every route, including models reached only through routing.
  • Priority decides between configurations. Content routes take their priority from their place in the list. Two overlapping routing rules cannot share a priority.
  • Data routes win over the route that would otherwise apply when their priority is equal or higher. They are checked on every message.
  • A destination that cannot serve the request's API (natively or by conversion) is skipped. If no destination of the route can serve it, the request uses normal routing.

Step 2: which model in the route​

Route typeHow the model is picked
Route toThe first target, in order, whose credential is enabled and that can serve the request's API. Later targets are used only when an earlier one fails (see Failover).
Routing groupIf the conversation is pinned to a member that is still valid, that member. Otherwise the member whose share of conversations (or tokens, in token mode) is furthest below its weight. This is deterministic, not random, so traffic converges on the configured split.
No routeThe only enabled provider that serves the model, or one picked at random among several and then pinned.

Members from the request's own API family are preferred. Members that can only be reached by conversion are used when no own-family member is available. Skipped members are left out and the remaining weights are rescaled.

Conversations and caching​

Providers cache the start of a prompt (system prompt, tools, earlier turns). The cache belongs to one provider credential and model, and every move to another model starts a cold cache: full input price and slower first tokens. So routing decisions are made per conversation, not per request.

A gateway that routes each request on its own spreads one conversation across providers and pays for a cold cache every time it moves:

Turn 1
Provider A
Cache written
Turn 2
Provider B
✗ Cold cache
Turn 3
Provider A
✗ Cold again
Turn 4
Provider C
✗ Cold again
QuilrAI

The LLM Gateway keeps the conversation where its cache is:

Turn 1
Route decided
Provider A
Cache written
Turn 2
Pinned
Provider A
✓ Cache hit
Turn 3
Pinned
Provider A
✓ Cache hit
Turn 4
Pinned
Provider A
✓ Cache hit
QuilrAI

Cache-aware by design​

Where a move could cost the cacheWhat the gateway does
Choosing a routeDecided on the first message and kept (Keep destination is the default).
Weighted splitsWeights split new conversations, not requests; each conversation stays on its member.
Several providers for one modelOne is picked once and pinned.
Writing the pinOnly after a successful response, so an error never pins a conversation somewhere new.
Complexity routingUpgrade when needed only moves up, so a conversation rebuilds its cache at most once per level.
FailoverThe conversation stays on the backup that answered, instead of moving back and paying for a second cold cache.
Provider-held stateRequests that continue a stored response or cached content are never moved to another credential.
Keeping caches aliveKeep prompt cache warm extends or refreshes caches without moving the conversation.
The request bodyNative requests are forwarded byte for byte, and cache markers are carried across format conversion, so the cached prefix still matches.
Helper modelsEach requested model is pinned on its own, so a coding tool's small model does not drag the main conversation along.

How the gateway identifies a conversation​

The first of these present on the request becomes the conversation key:

OrderSource
1Body previous_response_id (Responses)
2Header x-conversation-id, conversation-id or conversation_id; body metadata.conversation_id; Responses conversation
3The client's session id: the Claude Code session in metadata.user_id, metadata.session_id, client_metadata.session_id, then thread or run ids in metadata
4Body prompt_cache_key
5A hash of the user identity, tool definitions, system prompt and first user message

The key is also scoped to the Quilr key and the requested model, so a coding tool's small helper model is routed separately from its main model. Send a conversation ID header when you can: with option 5, two conversations from the same user that start identically share one key. The Identity & Network Trust card can require one.

Conversation pinning​

When a routing group or a multi-provider model picks a member, that choice is pinned for the conversation after the response succeeds (a stream has to finish). Failed requests never create a pin. A pin lasts until the conversation has been idle for 1 hour.

A pinned conversation is routed again when:

  • the configuration that chose it is edited (any change, including weights or priority), removed or expires,
  • the routing group or API changes,
  • the pinned provider is disabled, loses its credential, or can no longer serve the request's API,
  • 3 requests in a row fail on it with a 5xx, 401, 403, 404, 408, 424 or 429 (requests blocked by QuilrAI guardrails do not count),
  • a data route matches,
  • or failover moves it, in which case the conversation is pinned to the backup and does not move back.

Route to needs no pin: its first available target is the same on every request.

Conversation behaviour​

Each content route has a Conversation behaviour (called Switching in the complexity form). It decides whether a route is re-checked on later messages:

OptionWhat happensCache effect
Keep destination (default for most routes)The first message decides; later messages stay there. A higher route that re-checks every message can still take over. If a check could not finish on the first message, nothing is kept and the next message decides.One model for the conversation.
Upgrade when needed (default for complexity routes)Later messages can move the conversation up a level, never down.One cache rebuild per upgrade.
Re-check every messageEach message routes on its own. With AI judging, that is one AI check per message.Can lose the cache on any message.
Message 1: hi
Low
Haiku
Cache built
Message 2: design a ledger
Upgrade to High
Opus
Cache rebuilt once
Message 3: thanks
Stays High
Opus
Cache hit ✓
QuilrAI

Data routes always re-check every message and, once matched, keep their destination for the rest of the conversation until a higher-priority data route matches or the rule changes.

A stored destination is dropped and routing runs again when its rule is edited, removed or expires, or when the destination can no longer be used.

Keep prompt cache warm​

Provider caches expire after a few minutes without traffic. Conversations > Set up opens Keep prompt cache warm for chosen apps. Both options are off until you turn them on.

SettingWhat it does
Longer cache on Anthropic and Bedrock ClaudeSets a 1 hour TTL on cache breakpoints for Anthropic, Azure AI Foundry Anthropic and Claude 4.5+ on Bedrock. No extra requests; cache writes cost about 2x base input instead of 1.25x.
Refresh requests on OpenAI, Azure, Gemini and VertexAbout 60 seconds before the provider cache would lapse, the gateway resends the last prompt for 1 output token (16 on Responses). Applies to OpenAI, Azure OpenAI, OpenAI-compatible, DeepSeek, Gemini, Vertex and Responses. Each refresh is billed to the app, counted toward its quotas and spend, and logged as a cache-warm request. Refreshes stop on a quota refusal or after 2 upstream errors in a row.
Stop refreshes after idle (minutes)1 to 60, default 30.
Minimum prompt (tokens)Smaller prompts are not kept warm. Default 4,000.

Never kept warm: requests with server-side or hosted tools (including web search), Responses conversation or background state, requests translated to another API, private inference, blocked requests and response-cache hits, and prompts below the minimum. A refresh never moves or extends a conversation pin.

routing.keep_cache_warm = { stop_after_idle_minutes: 30, min_prompt_tokens: 4000, extend_cache_lifetime: true, refresh_requests: true };
note

The Response cache on the Token Savings card is different: it answers a repeat of a non-streaming request from an earlier response, without calling the model. Keep prompt cache warm only keeps the provider's prompt cache alive.

Routing types​

TypeDecides byUses AIDefault conversation behaviour
Route toScope only (app, group, requested model, ...)NoKeep destination
Routing groupsWeights across modelsNoPinned per conversation
Request is aboutA sentence describing the requestsYesKeep destination
Prompt complexityLow, Medium or High levelYes, or length onlyUpgrade when needed
Data detectedData types in the latest messageData scanChecked on every message, then kept
Text containsWords or phrasesNoKeep destination
ClientThe coding tool named in request headersNoKeep destination
Budget usedHow much of a budget the requester has usedNoChecked on every message (API routes); set it in a policy

All of them apply on the Chat Completions, Anthropic Messages, Responses, Vertex AI and Bedrock Runtime surfaces. OpenAI Realtime supports routing groups only. Embeddings are not routed.

Start from a recipe opens a form for common setups: Send an app to a model, Primary model with a backup, Split traffic across models, Route by topic, and Route sensitive data to a private model. Complexity recipes include Small model for simple prompts, Three tiers: small / medium / large, and By message length only.

Route to​

An ordered list of provider label and model pairs. The first target that is enabled, has a credential and can serve the request's API takes the request, whatever model the client asked for. The rest are its failover order.

  1. Choose Send an app to a model under Start from a recipe, then pick the apps or groups.
  2. Under Send to, keep Ordered targets. Pick a Provider label and a Model for target 1.
  3. Add target for each backup and drag to reorder.
  4. Save to draft, then Review & publish.
policy support_copilot_route priority 650 {
request where application.group_key == "support-copilot"
then { routing.targets = [
{ model: "gpt-4.1", provider_label: "openai_primary" },
{ model: "gpt-4.1", provider_label: "azureopenai_primary" }
]; }
}

Routing groups​

A weighted split across models. Weights total 100.

  1. Choose Split traffic across models under Start from a recipe, then pick who it is for.
  2. Under Split, give it a Name, choose the Kind (Chat completion, Anthropic messages, Responses, Vertex AI, Bedrock or Realtime) and the Mode:
    • New conversations: weights split new conversations.
    • Split by tokens: weights split total tokens.
  3. Add model for each model and set its weight. Balance evenly splits them equally.

Because a conversation stays on the model it started with, request and spend shares can differ from the weights. The group is saved as policy-<name> and never affects an app routing group with the same name.

Content routes​

Content routes pick the destination from what the conversation contains. They share one form, Add a content route: who it applies to, Match when, and Send to (Ordered targets or Routing group). A content route can only set a destination, its conversation behaviour, observe only and its own failover.

Request is about (description routing)​

Describe the requests in a sentence or two (up to 300 characters). An AI classifier judges whether the latest user message matches.

  • Long messages are classified on a bounded portion of the text; check long prompts with Test routing.
  • Up to 32 descriptions are checked per request, highest priority first.
  • With Keep destination, the classifier runs on the first message of a conversation; a result is reused while the latest user text is unchanged.
  • If the classifier is unavailable or times out, the route is skipped and the request falls through to the next route. The skip is shown in Routing activity.
policy route_coding priority 720 {
request where application.method_type == "chat"
and about(content.last_user_text, "Programming help: source code, SQL queries, shell scripts, bugs or stack traces.")
then { routing.targets = [{ model: "claude-sonnet-4-6", provider_label: "anthropic_primary" }]; }
}

Complexity routing​

Send simple prompts to a small model and hard ones to a large one. Start from a complexity quick add card or Set up complexity routing under Complexity levels.

  1. Application: pick the app, or leave empty for every app.
  2. When the app asks for (optional): limit to requested model names, for example auto.
  3. How to judge:
    • AI reads the message (falls back to length): the classifier compares the latest user message with your level descriptions. If it is unavailable or unsure, the length bounds decide.
    • Message length only: words or tokens in the latest user message decide. No AI call, no added latency or cost.
  4. Levels: for Low (and Medium with Use a Medium level) write what the level means and its upper bound in Words or Tokens. A message at or under the Low bound is Low, at or under the Medium bound is Medium, anything else is High.
  5. For each level choose Normal routing, One model or first available, or Split by %.
  6. Switching: see Conversation behaviour.
LevelDefault descriptionDefault bound
LowShort factual questions, greetings, or one-step rewrites of a few sentences.300 tokens
MediumEveryday drafting, summarising or analysing a short document or snippet.4,000 tokens
HighMulti-step reasoning, planning, large codebases or long documents.Anything above Medium

Without Complexity levels, Prompt complexity counts words over all user text in the request: 6 or fewer is Low, 7 to 10 is Medium, more is High.

policy cx priority 760 {
request where application.group_key == "support-copilot"
then { routing.complexity_levels = {
low: { description: "Greetings and short factual questions", max_tokens: 300 },
medium: { description: "Everyday drafting and summaries", max_tokens: 4000 },
high: { description: "Multi-step reasoning, design or long analysis" }
}; }
}
policy cx_low priority 759 {
request where application.group_key == "support-copilot" and content.context_complexity == "low"
then { routing.targets = [{ model: "claude-haiku-4-5-20251001", provider_label: "anthropic_primary" }];
routing.route_decision = escalate; }
}
// cx_medium and cx_high follow the same shape

For length only: { method: length, low: { max_words: 15 }, medium: { max_words: 120 }, high: {} }.

Data detected​

Move a conversation to a model you trust when chosen data types appear, for example an on-premises model for secrets. Choose Route sensitive data to a private model under Start from a recipe, choose the data types, then the destination.

  • Data routes are evaluated on every request and go first in the list by default.
  • Once matched, the conversation keeps this destination.
  • Best effort: if the scan cannot run, normal routing is used. Use Security guardrails to redact or block.
  • Types used only for routing do not create findings, redact or block. Matches appear only in Routing activity.

Text contains​

Route when the latest user message contains any of a list of words or phrases. Comma separated, not case-sensitive. * matches any text in between (refund*order), ? one character.

Client​

Route by the coding tool named in the request headers, for example Claude Code, Codex, Cursor, Cline or GitHub Copilot. The client reports this itself; it is not an identity check.

Observe only​

Every content route has an Observe only switch. The route is checked on every message and Routing activity records where matching conversations would go, while requests keep their normal destination. AI and data checks still run, so they still add latency. Observe only cannot be combined with a conversation behaviour or a failover setting.

Order​

Under Order, place a route 1st, after another route, or Last. Higher in the list wins. New routes go last; data routes go first. Earlier and Later on the card move a route.

Budget used​

Route by how much of a budget the requester has used, for example to a smaller model past 80% of their monthly spend. Write it in a policy with budget_used_pct("<budget id>"), compared with >=, >, <= or < against a number, or add conditions.budget to a route through the Routing and failover API. The value includes the requester's grants. In a policy, add routing.route_decision = per_message so it is checked on every message rather than kept for the conversation; API routes do this for you.

If the usage cannot be read, the rule does not match and normal routing applies. Route below 100%: a budget that is used up and enforced still refuses the request. Example: Move to a cheaper model as a budget runs low.

When a routing check fails​

Content and data routes are best-effort routing, not enforcement. A check that cannot complete never blocks the request: it is routed as if that route did not match. Use Security guardrails when data must be blocked or redacted.

Failover​

When the chosen provider fails, the gateway retries it or moves the request to a backup. Configure it under Failover defaults for everyone or a scope, or per route with Set failover for this route. A route's own failover setting wins; otherwise the highest-priority failover setting that matches applies; with none, the defaults below apply.

Settings​

SettingDefaultValues
Move failed requests to another modelOnOff: failed requests are only retried on the same model, for the errors in Fail over on.
Where it can go+ same model on other connectionsRoute models only, + same model on other connections, + other app models
Backup modelsNoneUp to 8 provider label and model pairs, tried after the route's own models.
Fail over onTimeouts, Connection errors, Rate limits (429), Provider errors (5xx)Opt in: Rejected credentials (401/403), Model not found (404), Prompt too long for the model, Provider content filter
Max tries31 to 3. Counts every send to a provider, retries included.
Same-model retries20 to 2. Only for errors in Fail over on.
Time budget (s)601 to 600. Only stops new tries from starting; a try in flight is never cut. Starts when the gateway first sends upstream, after its own guardrail checks.
Cut off slow tries / Per-try timeout (s)Off1 to 300.
Advanced > App model rankingNoneOrder for + other app models.
Advanced > Model capabilitiesNoneContext window, max output, tools, images, JSON schema and streaming per model.
routing.failover = { enabled: true, scope: same_model, fallbacks: [{ model: "gpt-4.1", provider_label: "azureopenai_primary" }],
on: [timeout, connection, rate_limit, server_error], max_attempts: 3, same_target_retries: 2,
total_timeout_ms: 60000, attempt_timeout_ms: 0 };

A partial value is filled from the defaults, not merged with other failover settings.

Where a failed request goes​

Backups are tried in tiers. A tier is used only when the tiers before it have no usable candidate left.

1. Rest of the route
Route to: next target in order
Group: other members, drawn by weight
2. Backup models
In the order listed
3. Same model, other connections
Default scope
Other provider labels serving the model
4. Other app models
Only with + other app models
Ranked, at most 8
QuilrAI

Every candidate must have an enabled credential, be allowed by Allowed Models, be inside a matched data route's destinations, and serve the request's API without dropping any of its features. A candidate that is the same provider account and model as one already tried is skipped. Before a backup is sent, its quotas and per-model limits are checked; a backup over its limit is skipped.

Other app models are ranked by App model ranking, then same API family, then same vendor, then closest price. Without Model capabilities, such a model is used only for requests with no tools, images or JSON schema, an estimated prompt under 8,000 tokens and requested output up to 4,096 tokens, and for streams only when the provider streams natively for the API.

Because Max tries is at most 3 and includes the first send, a request reaches at most 2 backups.

Retry or move​

  1. Was it a QuilrAI block or a gateway error?Guardrail, quota, access, conversion, configuration
    yes:
    Returned to the clientNever retried
  2. Are Max tries or the Time budget used up?
    yes:
    Last provider's error
  3. Is the error in Fail over on, and is a backup left?A 429 with Retry-After of 2 s or less retries in place instead
    yes:
    Next backupSame-model retry count resets
  4. Is it a timeout, connection error, 429 or 5xx in Fail over on, with same-model retries left?
    yes:
    Retry the same model0.5 s, then 1 s, or Retry-After up to 10 s
  5. None of the above
    The provider's error, unchanged
QuilrAI
  • Fail over on controls both moving and same-model retries: an error left out of it is returned to the client on the first try.
  • A retry is made only if its delay fits in the remaining time budget.
  • When every try fails, the client gets the last provider's error with its own status, body and Retry-After, exactly as if failover did not exist.
  • When nothing could be sent (every candidate marked down or over quota, or the time budget already used up), the client gets 503 all_targets_unavailable with x-quilr-error-source: gateway and, when known, Retry-After.
  • With Provider content filter on, a successful response that the provider refused or filtered is also sent to another provider.

Requests that never move​

These are only retried on the same provider:

  • requests that continue provider-held state: previous_response_id, Responses conversation or background, Gemini cachedContent, stored prompts, containers and MCP servers held by the provider,
  • requests with hosted or server-side tools, or that reference provider file ids,
  • requests that name a provider with provider, provider_label, X-Provider-Name or X-Provider-Label,
  • requests served by a private inference edge.

For stateful requests, a retry is made only when the provider certainly did not run the first try (the connection was never opened, or a 429). Embeddings fail over only to the same model on another connection. OpenAI Realtime and Assistants are not retried or failed over by the gateway.

Streaming​

For a stream, the gateway waits for the provider's first event before it starts responding to the client. An error, timeout or error event before that can still retry or fail over. Once the first event reaches the client, the stream is committed to that provider and a later error is passed through.

Provider health and cooldown​

The gateway tracks the health of every provider credential and model it calls:

FailureEffect
429Cooling down for the provider's Retry-After, or 10 s, between 1 and 120 s.
Timeout, connection error, 5xxAfter 3 failures within 60 s: cooling down for 30 s, doubling on each repeat up to 300 s (with jitter).
401, 403, 404Cooling down for 300 s.
Prompt too long, content filterNever.

A cooldown only affects requests whose Fail over on includes the error that caused it, and each kind of error cools down on its own: a 401 cooldown does not hide a later run of 5xx errors from requests that only fail over on 5xx.

During an outage you can also mark a provider down for every app, with an expiry, and reset cooldowns once it recovers, with the Incidents API.

A cooldown routes around a provider; it never refuses a request. A cooling target is skipped while a healthy backup exists. When its cooldown ends, one request at a time is sent to it as a probe; success clears it, failure starts a longer cooldown. If every candidate is cooling, including a route with a single target, the request is still sent, to the one that recovers first.

Failover, pinning and quotas​

  • After a request moves from A to B and succeeds, the conversation is pinned to B and stays there until it goes idle. There is no automatic failback.
  • A content-route decision stored for the conversation is not changed by a failover.
  • Quotas count one logical request. Each provider that was actually sent to is charged for its own usage at its own price; the app-level request count is charged once.

See what happened​

HeaderWhenValue
x-quilr-attemptsMore than one sendNumber of sends, retries included
x-quilr-failoverServed by a backupThe tier that served it: route, fallback, same_model or app

The request log lists every try with its provider, model, tier, outcome, status, latency and spend. The failover preview shows where an app's failed requests would go.

Test and verify​

Test before you publish​

Use Test routing in the Routing Groups & Fallbacks panel to check a routing scenario. For a broader policy check, open Test > Try a situation in the LLM Gateway Policy Engine, describe the request, and review the rules that decide its settings. Test both a request that should route and one that should keep its requested model.

Use Replay activity to compare the candidate rules with recent recorded activity before publishing. See Author, simulate and publish.

Check live traffic​

Every successful gateway response carries:

HeaderValue
x-quilr-routedefault when no route applied (including when a route's destination could not serve the request), otherwise the route id: r_ and 16 hex characters. The id never names the policy.
x-quilr-served-modelThe model that served the request. Omitted on deployments that hide model identity.
curl -sD - -o /dev/null https://guardrails-usa-2.quilr.ai/openai_compatible/v1/chat/completions \
-H "Authorization: Bearer sk-quilr-xxx" -H "Content-Type: application/json" \
-d '{"model": "gpt-4.1-nano", "messages": [{"role": "user", "content": "hi"}]}' | grep -i x-quilr

The panel's Activity tab shows per route and app over 24 h, 7 d or 30 d: conversations, requests, spend (including keep-warm refreshes), cache read, added time, failures, upgrades and fallbacks. Hover a route to see its route id.

Rules and limits​

  • The requested model must be allowed for the app. A name such as auto that exists only for routing is rejected with model_access_denied unless the app's allowed models include it.
  • Under the Policy Engine, a client that sends an app routing group name the policy does not define gets 403 policy_routing_group_not_defined, or policy_routing_group_wrong_surface when the group is for another API.
  • Two overlapping routing rules cannot share a priority.
  • Upgrade when needed is only for complexity routes. Observe only cannot be combined with a conversation behaviour or failover.
  • Request is about cannot be negated.
  • On deployments without GPU models, the AI classifier is never called; complexity routes use the length bounds.
  • A spend budget needs a price for every model it can reach, including failover backups; see Rate, token and budget limits.