SDK Mode
Check content and optimize text directly from your application code - no LLM proxy required.
Overview
SDK mode exposes a standalone content-checking endpoint (POST /sdk/v1/check) that you can call at any point in your pipeline. Instead of routing LLM traffic through the Quilr gateway, you call this endpoint yourself to scan messages, text, or structured JSON for sensitive data and adversarial inputs.
Request checks can also inspect inline images and documents using the same attachment scanning settings as proxied model requests. This includes document text, OCR and enabled Image Risks. Your application applies the returned allow, redact or block decision before continuing.
SDK attachment checks require a gateway deployment that includes this feature. Older deployments can return a successful text check without inspecting attached file content. Inspect the returned attachment coverage as well as the overall action.
For text optimization, use the separate POST /sdk/v1/token-savings endpoint. Select any of seven token-saving methods and receive the final text plus savings estimates. Its deterministic output helps keep repeated prompt content hash stable and cache friendly when you apply it consistently.
To test a key before integrating it into your app, open the LLM Gateway Playground and select Quilr SDK, then Guardrail check or Token savings. For checks, choose Text, Messages, or JSON and select the check type and hashing mode. For token savings, enter text and select your methods. Both operations generate matching cURL, Python and JavaScript.
Common uses:
- Check user input before forwarding to an LLM
- Scan LLM responses before returning them to users
- Scan file uploads, form fields, or other non-LLM content
- Integrate with a self-hosted LiteLLM proxy
- Optimize retrieved content, API results and tool output before sending them to your LLM
Authentication
SDK mode requires a dedicated SDK key - regular LLM proxy keys are rejected with 403.
Create an app in Settings > AI Gateway > LLM Gateway > Create App, choose App-only credentials and select the Quilr SDK tile. It provides standalone content checks and token savings, has no models, and cannot be combined with other providers. Use the app's Quilr key as a Bearer token:
Authorization: Bearer sk-quilr-xxx
Api-Key: sk-quilr-xxx is also accepted.
You can optionally include an X-User-Email header for identity-aware enforcement if that is configured on the app.
Request Format
POST /sdk/v1/check
Three input formats are supported. Send one content field: messages, text, or json.
Messages (conversation)
Send an OpenAI-style conversation. Request checks select recent user messages for message-text detection according to your key settings. Attachment discovery also includes supported files in the supplied conversation history.
Set type: "request" or "response" explicitly. The explicit value wins over message roles. Without it, a final assistant message selects a response check and returns the text response shape; otherwise the messages are checked as a request.
{
"messages": [
{"role": "user", "content": "..."},
{"role": "assistant", "content": "..."}
],
"type": "request",
"metadata": {}
}
Text (raw string)
Use this to check a single piece of text. Set type: "request" for input checks; omitting type checks raw text as a response.
{
"text": "some text to check",
"type": "request"
}
Images and documents (request checks)
Set type: "request" and send the file bytes inline. For a conversation, put image and file blocks inside a message's content array. Text is optional: an image-only or document-only message is valid.
{
"type": "request",
"messages": [{
"role": "user",
"content": [
{"type": "text", "text": "Inspect these attachments."},
{
"type": "image_url",
"image_url": {"url": "data:image/png;base64,<BASE64_IMAGE_BYTES>"}
},
{
"type": "file",
"file": {
"filename": "sample.pdf",
"file_data": "data:application/pdf;base64,<BASE64_PDF_BYTES>"
}
}
]
}]
}
Replace the placeholders with standard base64 of each original file's bytes. Recognized image/file fields can also carry bare base64. Anthropic-style image and document blocks with a base64 source are accepted, along with the gateway's other supported inline content blocks.
The SDK uses the same attachment discovery as normal gateway requests. Data URLs embedded in message strings, text content parts, HTML and tool-result text are inspected too, including wrapped base64. For a single string, use request-side text:
{
"type": "request",
"text": "Inspect <img src=\"data:image/png;base64,<BASE64_IMAGE_BYTES>\">"
}
A bare base64 string in ordinary text is treated as text, not inferred to be an image. Remote URLs and provider file IDs are not fetched. The request still needs messages or text; a standalone attachment array is not a fourth input format.
Enable Scan attachments in the SDK app's Attachments settings, or the equivalent published Policy Engine control. OCR and document text use the enabled request-side text detections. Visual classification also needs Classify images and at least one enabled Image Risks type. See supported files and coverage.
This protection is request-side only. Response checks do not scan generated images or documents, and explicit json input remains a check of JSON values rather than an attachment envelope.
JSON (structured content)
Put the value to scan in json. Pass an actual JSON value, rather than a serialized object inside text.
{
"type": "request",
"json": {
"customer": {
"firstName": "Alex",
"fullName": "Alex Morgan",
"name": "ALEX"
},
"contacts": [{"email": "alex@example.com"}],
"active": true,
"notes": null
},
"hashing_mode": "case_insensitive",
"metadata": {"source": "customer-profile"}
}
json accepts objects, arrays, and scalar roots, including strings, numbers, booleans, and null. JSON checks default to type: "request"; use "response" to apply response-side scopes. JSON-looking content in text stays in text mode.
String and number values are scanned together as one document. Field and parent names provide context, including readable camelCase and snake_case labels. Keys are preserved and entity detections found only in keys are discarded. Key names do not force detection: your configured categories, sensitivities, scopes, and actions still apply. For example, the SDK's labeled-name fallback requires the NAME subcategory sensitivity to be high.
A detected entity can match occurrences in multiple values, including under different keys. This is document-wide entity detection, not independent classification of each field.
Request parameters
Response
Completed checks return HTTP 200, including content blocked by guardrails. Authentication, validation, and policy-access errors use non-2xx responses. The response shape depends on the input and check type.
Use action for application control flow. Message-text and JSON findings appear
in predictions; attachment findings and scan coverage appear in attachments.
Messages response
{
"status": "safe | redacted | blocked",
"action": "allow | redact | block",
"messages": [...],
"blocked_text": "...",
"predictions": [
{
"id": "...",
"name": "...",
"type": "redact",
"sensitive_entities": ["123-45-6789"],
"entity_texts_with_subcategories": {
"123-45-6789": "SOCIAL SECURITY NUMBER"
}
}
],
"categories_detected": ["pii", "email", "ssn"],
"similar_entities": [],
"similar_entities_truncated": false,
"placeholder_masking": {
"hashing_mode": "case_sensitive",
"text": "My SSN is <qe-01a54629efb95228>.",
"messages": [
{ "role": "user", "content": "My SSN is <qe-01a54629efb95228>." }
],
"placeholders": [
{
"placeholder": "<qe-01a54629efb95228>",
"value": "123-45-6789",
"sub_category": "SOCIAL SECURITY NUMBER",
"category_id": "data_risk_category_pii",
"action": "redact",
"message_index": 0
}
]
},
"error": {...}
}
messages- the (possibly redacted) messages array;nullif blockedblocked_text- only present whenstatusisblockedpredictions- rule-level details, including exactsensitive_entitiesandentity_texts_with_subcategorieserror- only present whenstatusisblocked
Text response
{
"status": "safe | redacted | blocked",
"action": "allow | redact | block",
"original_text": "...",
"processed_text": "...",
"predictions": [
{
"id": "...",
"name": "...",
"type": "redact",
"sensitive_entities": ["555-867-5309"],
"entity_texts_with_subcategories": {
"555-867-5309": "PHONE NUMBER"
}
}
],
"categories_detected": ["pii", "phone"],
"similar_entities": [],
"similar_entities_truncated": false,
"placeholder_masking": {
"hashing_mode": "case_sensitive",
"text": "Call me at <qe-59c0b4a6fc3c3b2c>.",
"messages": null,
"placeholders": [
{
"placeholder": "<qe-59c0b4a6fc3c3b2c>",
"value": "555-867-5309",
"sub_category": "PHONE NUMBER",
"category_id": "data_risk_category_pii",
"action": "redact"
}
]
},
"error": {...}
}
processed_text- the redacted text;nullif blockedpredictions- rule-level details, including exactsensitive_entitiesandentity_texts_with_subcategorieserror- only present whenstatusisblocked
Attachment results
When request attachments are recognized, the response adds attachments with scanned, summary and items. This describes file coverage and findings separately from message-text predictions. It contains metadata, not the file bytes or extracted text. The paths below are relative to attachments.
Monitor keeps the original inline content. Redact, Partial redact or Tokenize removes the whole affected attachment, rather than editing its pixels or document contents. The returned messages, processed_text and applicable placeholder view reflect that removal. An attachment-only removal still produces status: "redacted", action: "redact", even if the surrounding message text is unchanged.
Attachment blocks, including file type and configured limit blocks, use the existing HTTP 200 SDK verdict: status: "blocked", action: "block" and an error with the attachment code. Do not forward content after a block. Attachment errors do not appear as text entity placeholders.
The SDK also recognizes the gateway's supported media in request extension fields. When an attachment is outside the primary messages or text field, the response adds processed_body: the complete sanitized input body, with primary text redactions applied too. It is null on a block. Use that body to retain rewritten extension fields; replacing only messages would not apply a removal elsewhere in the input.
Scanner errors, timeouts and unfinished scans fail open. They can return action: "allow" while the attachment metadata reports incomplete coverage. An unreadable-file policy is a separate control. Check the per-file status and flags, and see Actions and unreadable files.
JSON response
This example assumes NAME is detected and its configured action is redact:
{
"status": "redacted",
"action": "redact",
"original_json": {"name": "alex", "active": true, "notes": null},
"processed_json": {"name": "XXXXXXXX", "active": true, "notes": null},
"original_text": "{\"name\": \"alex\", \"active\": true, \"notes\": null}",
"processed_text": "{\"name\": \"XXXXXXXX\", \"active\": true, \"notes\": null}",
"predictions": [
{
"id": "data_risk_category_pii",
"type": "redact",
"sensitive_entities": ["alex"],
"entity_texts_with_subcategories": {"alex": "NAME"},
"entity_json_paths": {"alex": ["/name"]}
}
],
"categories_detected": ["pii"],
"placeholder_masking": {
"hashing_mode": "case_sensitive",
"json": {"name": "<qe-4135aa9dc1b842a6>", "active": true, "notes": null},
"text": "{\"name\": \"<qe-4135aa9dc1b842a6>\", \"active\": true, \"notes\": null}",
"messages": null,
"placeholders": [
{
"placeholder": "<qe-4135aa9dc1b842a6>",
"value": "alex",
"sub_category": "NAME",
"category_id": "data_risk_category_pii",
"action": "redact",
"json_paths": ["/name"]
}
]
},
"similar_entities": [],
"similar_entities_truncated": false
}
original_json,processed_json, andplaceholder_masking.jsonare actual JSON values. The corresponding*_textfields contain serialized JSON for compatibility.processed_jsonfollows the configured redact or partial-redact action. SDK Mode has no response to restore, so a Tokenize action is applied as redact. Safe and monitor-only values stay unchanged. Keys, nesting, array order, empty containers, booleans, andnullare preserved. A masked number becomes a string; unchanged numbers keep their numeric type.- Blocked results have
action: "block",processed_json: null,processed_text: null, anderrordetails. Placeholder output remains available for inspection and does not override the block. Checkactionbefore consuming content;nullcan also be a valid safe JSON root. entity_json_pathsmaps each detected entity to matching value locations. Placeholder entries includejson_pathsfor replaced locations. These are JSON Pointers:/contacts/0/emailaddresses an array item,~is escaped as~0,/as~1, and""means the root.
Placeholder masking and hashing mode
Every completed check includes placeholder_masking, an additional view with full placeholders for entities whose effective action is redact, partial-redact, or block. Monitor-only values remain unchanged. Use placeholder_masking.messages for request messages, .text for raw text, or .json for JSON input. Text and JSON checks set .messages to null.
Tokens have the format <qe-{16-char-hash}>. The same source value and hashing mode produce the same placeholder, so the identifier for that value stays stable. placeholders[] retains original values and detection metadata so your application can restore values using the mapping.
The selected mode is echoed in placeholder_masking.hashing_mode. It applies to all placeholder entity types and changes hashing only; detection, ordinary redaction, and policy actions are unaffected. Whitespace, punctuation, and accents remain part of the hash input.
Repeated identical JSON values share a placeholder entry with every replaced path. In case-insensitive mode, different source spellings can share a token but retain separate mapping entries with their exact value and json_paths or message_index. Preserve the location and source spelling if you need to restore exact capitalization: the shared token alone cannot distinguish it.
Similar entities
Every completed check also includes similar_entities and similar_entities_truncated. These are response fields, not request parameters. When both names in {"first": "Alex", "full": "Alex Morgan"} are detected, a case-insensitive check can return:
{
"similar_entities": [
{
"relationship": "possible_name_match",
"reason": "name_contains",
"entities": [
{
"value": "Alex",
"hash": "4135aa9dc1b842a6",
"placeholder": "<qe-4135aa9dc1b842a6>",
"category_id": "data_risk_category_pii",
"sub_category": "NAME",
"json_paths": ["/first", "/full"]
},
{
"value": "Alex Morgan",
"hash": "60705d930a9ed84f",
"placeholder": "<qe-60705d930a9ed84f>",
"category_id": "data_risk_category_pii",
"sub_category": "NAME",
"json_paths": ["/full"]
}
]
}
],
"similar_entities_truncated": false
}
Pairs are advisory textual similarities, not verified identity matches, and do not merge hashes or change detection or masking. Only detected person-name subcategories within the same category are compared; passwords, identifiers, emails, and usernames are excluded. Names that were not detected are not inferred.
Matching does not use fuzzy spelling, initials, or partial-word matches such as Ann/Anna. Pairs do not imply transitive identity groups. Similarity normalization does not change the hash input: hashes always follow hashing_mode, and short and full names keep separate hashes.
Entities retain actual source spellings and json_paths for JSON or message_indices for messages. JSON keys are excluded. Monitor-only or overlapping name detections may appear, so a reported hash need not appear in the masked output. With no matching names the list is [].
Candidate values and returned pairs are capped at 256 each. similar_entities_truncated: true signals either limit; this limit affects only these advisory results, not detection or masking.
Validation errors
Check the HTTP status before reading a check result. Validation errors use an error object with message, type: "invalid_request_error", and code.
Code Examples
The examples use US East (guardrails-usa-2.quilr.ai); replace it with your region's host from the integration guide.
Image and PDF request check - Python
This sends local file bytes directly to the SDK check. It does not make a model call. Configure the SDK app's attachment protections and data types before testing.
import base64
import os
from pathlib import Path
import requests
def file_data_url(path, mime):
encoded = base64.b64encode(Path(path).read_bytes()).decode("ascii")
return f"data:{mime};base64,{encoded}"
response = requests.post(
"https://guardrails-usa-2.quilr.ai/sdk/v1/check",
headers={"Authorization": f"Bearer {os.environ['QUILR_SDK_API_KEY']}"},
json={
"type": "request",
"messages": [{
"role": "user",
"content": [
{"type": "image_url", "image_url": {
"url": file_data_url("sample.png", "image/png"),
}},
{"type": "file", "file": {
"filename": "sample.pdf",
"file_data": file_data_url("sample.pdf", "application/pdf"),
}},
],
}],
},
timeout=60,
)
response.raise_for_status()
check = response.json()
print(check.get("attachments"))
if check["action"] == "block":
raise RuntimeError(check["error"]["message"])
checked_messages = check["messages"] # Use these, not the original messages.
For a proxied model request with the same image/file blocks, see Send images and documents.
Structured JSON - Python
Send the JSON value without stringifying it. Check the action first and read processed_json directly, including when the safe result is null, false, 0, or an empty value.
import requests
response = requests.post(
"https://guardrails-usa-2.quilr.ai/sdk/v1/check",
headers={"Authorization": "Bearer sk-quilr-xxx"},
json={
"type": "request",
"json": {"name": "Alex", "active": True, "notes": None},
"hashing_mode": "case_insensitive",
"metadata": {"source": "customer-profile"},
},
timeout=15,
)
response.raise_for_status()
result = response.json()
if result["action"] == "block":
raise ValueError(f"Blocked: {result['categories_detected']}")
safe_json = result["processed_json"]
placeholder_json = result["placeholder_masking"]["json"]
possible_name_matches = result["similar_entities"]
Structured JSON - JavaScript
const response = await fetch("https://guardrails-usa-2.quilr.ai/sdk/v1/check", {
method: "POST",
headers: {
Authorization: "Bearer sk-quilr-xxx",
"Content-Type": "application/json",
},
body: JSON.stringify({
type: "request",
json: { name: "Alex", active: true, notes: null },
hashing_mode: "case_insensitive",
}),
});
if (!response.ok) throw new Error(`Check failed: ${response.status}`);
const result = await response.json();
if (result.action === "block") throw new Error("Content blocked");
const safeJson = result.processed_json;
const placeholderJson = result.placeholder_masking.json;
const possibleNameMatches = result.similar_entities;
Structured JSON - cURL
curl -X POST https://guardrails-usa-2.quilr.ai/sdk/v1/check \
-H "Authorization: Bearer sk-quilr-xxx" \
-H "Content-Type: application/json" \
-d '{
"type": "response",
"json": {"name": "Alex", "active": true, "notes": null},
"hashing_mode": "case_insensitive"
}'
Python - httpx (async)
A typical pattern: check the user message before sending it to your LLM, then check the LLM response before returning it to the user.
import httpx
QUILR_BASE = "https://guardrails-usa-2.quilr.ai"
QUILR_SDK_KEY = "sk-quilr-xxx"
async def check_messages(messages: list[dict]) -> dict:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{QUILR_BASE}/sdk/v1/check",
headers={"Authorization": f"Bearer {QUILR_SDK_KEY}"},
json={"messages": messages, "type": "request"},
timeout=5,
)
resp.raise_for_status()
return resp.json()
async def check_text(text: str, type_: str = "response") -> dict:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{QUILR_BASE}/sdk/v1/check",
headers={"Authorization": f"Bearer {QUILR_SDK_KEY}"},
json={"text": text, "type": type_},
timeout=5,
)
resp.raise_for_status()
return resp.json()
# --- Usage ---
import asyncio
from openai import AsyncOpenAI
openai = AsyncOpenAI(api_key="sk-openai-xxx")
async def safe_chat(user_message: str) -> str:
messages = [{"role": "user", "content": user_message}]
# 1. Check input
result = await check_messages(messages)
if result["status"] == "blocked":
raise ValueError(f"Input blocked: {result['categories_detected']}")
if result["status"] == "redacted":
messages = result["messages"] # use redacted version
# 2. Call LLM
response = await openai.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
)
reply = response.choices[0].message.content
# 3. Check output
result = await check_text(reply, type_="response")
if result["status"] == "blocked":
raise ValueError(f"Response blocked: {result['categories_detected']}")
if result["status"] == "redacted":
reply = result["processed_text"]
return reply
asyncio.run(safe_chat("What is my SSN?"))
Python - requests (sync)
import requests
QUILR_BASE = "https://guardrails-usa-2.quilr.ai"
QUILR_SDK_KEY = "sk-quilr-xxx"
def check_text(text: str, type_: str = "response") -> dict:
resp = requests.post(
f"{QUILR_BASE}/sdk/v1/check",
headers={"Authorization": f"Bearer {QUILR_SDK_KEY}"},
json={"text": text, "type": type_},
timeout=5,
)
resp.raise_for_status()
return resp.json()
# Check a piece of text before storing or displaying it
result = check_text("My credit card is 4111 1111 1111 1111", type_="request")
match result["status"]:
case "safe":
print("No issues found")
case "redacted":
print("Cleaned text:", result["processed_text"])
case "blocked":
print("Blocked. Detected:", result["categories_detected"])
JavaScript / TypeScript - fetch
const QUILR_BASE = "https://guardrails-usa-2.quilr.ai";
const QUILR_SDK_KEY = "sk-quilr-xxx";
async function checkMessages(messages: Array<{ role: string; content: string }>) {
const res = await fetch(`${QUILR_BASE}/sdk/v1/check`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${QUILR_SDK_KEY}`,
},
body: JSON.stringify({ messages, type: "request" }),
signal: AbortSignal.timeout(5000),
});
if (!res.ok) throw new Error(`Quilr error: ${res.status}`);
return res.json();
}
async function checkText(text: string, type: "request" | "response" = "response") {
const res = await fetch(`${QUILR_BASE}/sdk/v1/check`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${QUILR_SDK_KEY}`,
},
body: JSON.stringify({ text, type }),
signal: AbortSignal.timeout(5000),
});
if (!res.ok) throw new Error(`Quilr error: ${res.status}`);
return res.json();
}
// Example: guard a chat endpoint
async function safeChat(userMessage: string): Promise<string> {
const messages = [{ role: "user", content: userMessage }];
const inputResult = await checkMessages(messages);
if (inputResult.status === "blocked") {
throw new Error(`Blocked: ${inputResult.categories_detected.join(", ")}`);
}
const checkedMessages =
inputResult.status === "redacted" ? inputResult.messages : messages;
// ... call your LLM with checkedMessages ...
const llmReply = "...";
const outputResult = await checkText(llmReply, "response");
if (outputResult.status === "blocked") {
throw new Error(`Response blocked: ${outputResult.categories_detected.join(", ")}`);
}
return outputResult.status === "redacted" ? outputResult.processed_text : llmReply;
}
cURL
# Check raw text
curl -X POST https://guardrails-usa-2.quilr.ai/sdk/v1/check \
-H "Authorization: Bearer sk-quilr-xxx" \
-H "Content-Type: application/json" \
-d '{"text": "Call me at 555-867-5309", "type": "request"}'
# Check a conversation
curl -X POST https://guardrails-usa-2.quilr.ai/sdk/v1/check \
-H "Authorization: Bearer sk-quilr-xxx" \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "What is the password for admin@acme.com?"}
],
"type": "request"
}'
TrueFoundry Integration
Use QuilrAI directly as a TrueFoundry custom guardrail for LLM inputs and outputs. Configure Mutate, Custom Bearer Auth with a quilr_sdk key, and the dedicated POST /sdk/v1/check/truefoundry endpoint. TrueFoundry sends its own request/response envelope and consumes the returned verdict, transformed, and result fields.
See TrueFoundry Integration for dashboard setup, selecting input/output rails, complete request examples, streaming requirements, and error handling.
LiteLLM Guardrails Integration
If you run a self-hosted LiteLLM proxy, you can add QuilrAI as a native LiteLLM guardrail. The plugin calls /sdk/v1/check for you on requests, responses, tool calls and MCP tool calls. No changes are needed in your application code.
At a glance (QuilrUnifiedGuardrail):
- Supported: prompts and answers on OpenAI Chat Completions, Anthropic Messages, the Responses API, text completion and Gemini; embeddings; image generation prompts; text to speech; rerank queries; tool calls; MCP tool calls (all tools, or only the ones you choose); streaming; guardrail records in Langfuse and other LiteLLM logging integrations; fail-open or fail-closed
- Not checked: images, tool definitions sent to the model, and transcribed audio text (a LiteLLM limit)
Details are in What is supported and Limits.
Which class to use
The plugin file has two classes. Use QuilrUnifiedGuardrail for new setups.
QuilrUnifiedGuardrail uses LiteLLM's unified guardrail interface, so LiteLLM extracts the text from each API format and the plugin checks it. Tested with LiteLLM 1.106.2.
Prerequisites
- A Quilr guardrails API key (a
quilr_sdkkey), from either Quilr-hosted guardrails or a self-hosted Quilr guardrails deployment - Download quilr_litellm_guardrails.py and place it in the same directory as your LiteLLM
config.yaml
LiteLLM resolves custom guardrails by module path relative to the proxy working directory, so the plugin is provided as a single file placed alongside config.yaml. There is no package to install.
Environment variables
QUILR_GUARDRAILS_KEY=sk-quilr-XXXXXXXXX
QUILR_GUARDRAILS_BASE_URL=<QUILR GUARDRAILS BASE URL>
QUILR_GUARDRAILS_TIMEOUT=3 # Optional: seconds before a check times out (default: 3)
QUILR_GUARDRAILS_FAIL_CLOSED=false # Optional: block when Quilr can't be reached (QuilrUnifiedGuardrail only)
QUILR_GUARDRAILS_SCAN_MCP_TOOLS=all # Optional: which MCP tools to check (QuilrUnifiedGuardrail only)
What is supported
All of this applies to QuilrUnifiedGuardrail.
Modes
Set the mode per guardrail in config.yaml. You can add the same class several times with different modes.
pre_call or during_call? during_call adds no latency, but the LLM has already received the prompt when the check finishes, so it can only block, not redact. Use pre_call to stop data before it reaches the LLM.
API formats and endpoints
What content is checked
Streaming
With post_call, streamed answers are checked while they stream: every few chunks, and once at the end. A block ends the stream cleanly with the block message (finish_reason: content_filter in OpenAI format). Optional settings, under litellm_params:
Which setup to use:
- Sensitive values must be redacted:
streaming_transform_mode: incremental_diff - Block only, and nothing may reach the client before the check:
streaming_buffer_until_moderated: true - Fewest Quilr calls (monitoring):
streaming_end_of_stream_only: true, or a higherstreaming_sampling_rate
Only incremental_diff redacts while streaming. In the other setups, if Quilr redacts, LiteLLM drops the redaction and the client gets the original text. Use them to block.
Logging and tracing
Each check adds a guardrail record to LiteLLM's logs, so it shows in Langfuse, Datadog and the other LiteLLM logging integrations:
guardrail_provider:quilrguardrail_status:success,guardrail_intervened(blocked),guardrail_flagged(categories found, allowed),guardrail_failed_to_respond(Quilr unreachable) ornot_run(MCP tool skipped byQUILR_GUARDRAILS_SCAN_MCP_TOOLS)guardrail_response: Quilr's status, action and categoriesmasked_entity_count: how many checks redacted each category
The record never contains the checked content.
To group checks by conversation and user in Quilr, the plugin sends:
X-Conversation-Id: the request'smetadata.session_id(or LiteLLM's session ID)X-User-Email: the first value that looks like an email frommetadata.trace_user_id, the request'suser, the end user, or the LiteLLM key owner's email. If Quilr rejects the email (domain not allowed on the app), the check is retried without it- The session and trace IDs in the check's metadata, to match Quilr logs with LiteLLM or Langfuse traces
When Quilr can't be reached
By default the plugin fails open: if Quilr times out or returns an error, the content passes unchecked, a warning is logged, and the logging record shows guardrail_failed_to_respond. To block instead, set quilr_fail_closed: true on each guardrail that should fail closed, or QUILR_GUARDRAILS_FAIL_CLOSED=true for all of them. Blocked requests return HTTP 503.
If you use fail-closed, consider a higher QUILR_GUARDRAILS_TIMEOUT so slow responses aren't blocked.
Choosing which MCP tools to check
By default every MCP tool is checked. To check only some, set QUILR_GUARDRAILS_SCAN_MCP_TOOLS for all guardrails, or quilr_scan_mcp_tools under one guardrail's litellm_params.
For example, with tools read_logs (read-only), run_sql (read-only), delete_file (destructive), send_email (destructiveHint: false) and get_time (no labels):
- Applies to
pre_mcp_call,during_mcp_call,post_mcp_calland the tool list check. - Give tool names as the MCP server names them (
run_sql) or with LiteLLM's server prefix (myserver-run_sql). Names are case-sensitive. Label words are not, and can be written with-or_(read-only,write_only). Inconfig.yamlthe value can also be a YAML list. - A tool without labels counts as
writeonlyanddestructive(the MCP spec's defaults), so it is still checked by those. - The labels are read from the MCP server's tool list and cached for 5 minutes (a failed read is retried after 30 seconds). If the list can't be read, the tool counts as unlabeled.
- MCP servers that need each caller's own login (a per-user token; OAuth servers are expected to behave the same): the plugin can't read their tool list, so their tools count as unlabeled.
writeonlyanddestructivethen check all of them, andreadonlychecks none. Use tool names for these servers. Servers whose login is set in the LiteLLM config work normally. - In
during_mcp_call, LiteLLM doesn't pass the server name, so the labels are looked up on every MCP server. A tool name that exists on several servers counts as read-only only if it is read-only on all of them. nonetogether with other values is treated asall, with a warning in the log.- A skipped tool passes through unchanged. Its logging record has
guardrail_status: not_run.
The labels are set by the MCP server itself and are not verified. Read-only tools such as SQL or log readers often return the most sensitive data, so skip them only when you are sure.
LiteLLM config.yaml
Input and output guardrails
guardrails:
- guardrail_name: "quilr-input"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "pre_call"
default_on: true
- guardrail_name: "quilr-output"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "post_call"
default_on: true
Input guardrail with lower latency (during_call, block only)
guardrails:
- guardrail_name: "quilr-input-duringcall"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "during_call"
default_on: true
Redaction while streaming
guardrails:
- guardrail_name: "quilr-output"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "post_call"
default_on: true
streaming_transform_mode: "incremental_diff"
MCP tool calls (LiteLLM as an MCP gateway)
guardrails:
- guardrail_name: "quilr-mcp-input"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "pre_mcp_call"
default_on: true
- guardrail_name: "quilr-mcp-output"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "post_mcp_call"
default_on: true
Block when Quilr can't be reached
guardrails:
- guardrail_name: "quilr-input"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "pre_call"
default_on: true
quilr_fail_closed: true
For an existing QuilrGuardrail setup, use guardrail: quilr_litellm_guardrails.QuilrGuardrail with the pre_call, during_call or post_call modes.
Optional filtering
You can limit which requests have guardrails applied using these environment variables:
# Only apply guardrails to specific models (comma-separated)
APPLY_QUILR_GUARDRAILS_FOR_MODELS=gpt-4,gpt-4o,claude-3-opus
# Only apply guardrails to specific API key names (comma-separated)
APPLY_QUILR_GUARDRAILS_FOR_KEY_NAMES=production-key,user-facing-key
If neither variable is set, guardrails apply to all requests. If both are set, a request must match both filters (AND logic) for guardrails to be applied.
- LiteLLM gives MCP checks the model name
mcp-tool-call, soAPPLY_QUILR_GUARDRAILS_FOR_MODELSskips MCP checks unless you addmcp-tool-callto the list. - When
APPLY_QUILR_GUARDRAILS_FOR_KEY_NAMESis set, requests on routes where LiteLLM does not pass the key name (for example/v1/messages) are not checked.
Enabling guardrails per request
With default_on: true, guardrails run on every request. If you set default_on: false, pass the guardrail names in the request body instead:
curl -X POST http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-litellm-xxx" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello"}],
"guardrails": ["quilr-input-duringcall", "quilr-output"]
}'
Behavior summary
Limits
Not checked by the plugin:
- Images in prompts (Quilr checks text only)
- Tool definitions sent to the model (the
toolslist)
Limits that come from LiteLLM:
- Redaction while streaming works only in the OpenAI Chat Completions format. Other formats can block a stream but not redact it.
- Anthropic Messages (
/v1/messages) answers: LiteLLM does not apply changed tool calls, so tool-call arguments can be blocked but not redacted. - Audio transcription (
/v1/audio/transcriptions): LiteLLM's proxy does not run post-call guardrails on transcriptions, so the transcribed text is not checked. - Rerank: only the query is checked.
- MCP tool labels (
readonly,writeonly,destructiveinQUILR_GUARDRAILS_SCAN_MCP_TOOLS): LiteLLM does not pass them to guardrails, so the plugin reads them from the server's tool list. For servers that need each caller's own login this isn't possible; use tool names there. See Choosing which MCP tools to check.