LLM Gateway Policies
Govern data, tools, models, identity, spend and quality on every model call.
Two decision points per call
The document is evaluated twice for a single model call.
Controls that shape the outbound call run on request. Controls that judge what came back run on response. Data inspection runs on both: on request it catches what your users send, on response it catches what the model returns.
What you can match on
The twelve control surfaces
Every setting belongs to exactly one card, and the workspace is those twelve cards. Monitor, redact and block are settings inside a card, never separate cards.
Protecting data
Block secrets everywhere
runs on requestpriority 900
No scope condition, so it covers every request the gateway sees. A high priority keeps it above narrower, more permissive rules.
Redact personal data in responses
runs on responsepriority 750
Use partial-redact instead to mask only part of a value, the usual choice for
financial data where the last four digits still need to be readable.
Introduce a detection safely
runs on requestpriority 200
Publish on monitor, read a week of activity, then raise it. A low priority
keeps it clear of real enforcement rules.
Stop prompt attacks at ingress
runs on requestpriority 1000
Adversarial detections are ordinary data types, so prompt-attack defence has the same shape as secrets defence.
Several data types, several actions
A configuration on the Data & Adversarial Risks card holds as many data rules as you need. Each rule picks its own data types and its own action; they all share the configuration's scope and priority. Add data rule adds another.
runs on requestpriority 700
One request carrying an Aadhaar number, a customer name and an API key has the Aadhaar redacted, the name left alone but recorded, and the whole call refused because of the key. Adding a rule copies the scope from the rule above it, so you only pick the data types and the action.
redact and partial-redact rewrite only the findings their own data found
condition selected. The Aadhaar rule redacts Aadhaar numbers; it does not touch
the name beside them. A monitor rule leaves its own findings alone and cannot
suppress another rule's redaction.
A finding that no rule selects is left unchanged. Where two rules select the same finding, the higher policy priority wins, and at equal priority the more restrictive action wins. Rules inside one configuration share a priority, so they never compete with each other.
block stops the whole request rather than stripping the finding out of it.
redact, partial-redact and monitor all operate finding by finding.
Use separate configurations instead when the rules need different scopes or priorities, for example one for an application and another for a Smart Group exception. Rules that share a scope belong in one configuration.
Governing tool calls
Tool Controls decides whether a call may proceed at all, separately from what data it carries, so an agent keeps its read tools while losing its dangerous ones.
runs on requestpriority 950
runs on requestpriority 925
is set is a presence test needing no value. It narrows the rule to calls
carrying a tool invocation, leaving ordinary chat traffic alone.
Denying access
Gateway Access rejects the whole call before any model is contacted: no tokens spent, no provider round trip, and the data rules never run because the request never proceeds.
Deny a person completely
runs on requestpriority 1000
One condition, no application scope, so these people are refused on every gateway application, every model and every method. Use this shape while a leaver's credentials are still being revoked upstream.
Deny a group for particular models
runs on requestpriority 800
Interns keep full gateway access and are refused only when they reach for an
expensive frontier model. Swap the group row for User email is any of to do
the same for one person.
The allow-list alternative
runs on requestpriority 700
Same intent, opposite construction. The card above denies three named models and must be edited every time a new frontier model appears; this one names the two models Interns may use, so anything new is excluded by default. Prefer this shape unless you specifically need the denial recorded as a blocked call.
Everyone except
runs on requestpriority 850
A negated membership test turns one rule into a default-deny for an application. Anybody outside Finance Platform is refused, and new joiners are covered the moment they are added to the group.
Choosing how to stop a call
Runtime, routing and limits
Several cards can contribute to one configuration, giving a whole operating profile in a single sentence.
runs on requestpriority 700
matches pattern with *code* covers any model whose name contains "code", so
a newly released coding model inherits the whole profile with no policy change.
runs on requestpriority 850
Keyed on your own request metadata, so production gets a perimeter that development never sees: no separate application, no duplicated settings.
Budgets and spend
Budgets are a structured setting, so the card opens a form rather than a single value. Each budget in the list applies independently.
runs on requestpriority 500
Budget 1 gives every person their own 500 USD monthly allowance; Budget 2 caps the whole application at 10M tokens a rolling week. Both must hold, so an individual staying under budget can still be stopped by the team cap.
Usage is tracked against the ID. Keep it when changing the amount and recorded usage carries over. Changing the measure, period, timezone or grouping after publishing needs a new unique ID and starts a fresh count.
Spend budgets require input and output prices in USD per 1 million tokens for every matching provider and model. Requests without a matching price are blocked. Set prices once under Settings, Models.
Token savings and Prompt Store
runs on requestpriority 300
Savings show up as tokens saved in the activity view, so you can prove the reduction rather than assume it. See Token Saving for the cross-product guide.
runs on requestpriority 600
Production traffic must use a reviewed system prompt while development traffic stays free to experiment. See Prompt Store.
Response quality
runs on responsepriority 500
The threshold is the confidence at which a response counts as a hallucination.
Start at monitor, then move either the action or the threshold.
Scoping to people and groups
Every card carries one-click scope shortcuts, which add the condition and lift the priority so the narrower scope wins automatically.
No conditions means everyone. An individual exception therefore outranks a group rule without you choosing numbers.
runs on requestpriority 800
An exception layered above a stricter default. If the tenant-wide PII rule redacts at priority 750, this monitors at 800, so Support Tier 2 sees unredacted data in the Support Copilot application only.