Skip to main content

SDK Mode

Check content and optimize text directly from your application code - no LLM proxy required.

Overview​

SDK mode exposes a standalone content-checking endpoint (POST /sdk/v1/check) that you can call at any point in your pipeline. Instead of routing LLM traffic through the Quilr gateway, you call this endpoint yourself to scan messages, text, or structured JSON for sensitive data and adversarial inputs.

Request checks can also inspect inline images and documents using the same attachment scanning settings as proxied model requests. This includes document text, OCR and enabled Image Risks. Your application applies the returned allow, redact or block decision before continuing.

Gateway availability

SDK attachment checks require a gateway deployment that includes this feature. Older deployments can return a successful text check without inspecting attached file content. Inspect the returned attachment coverage as well as the overall action.

For text optimization, use the separate POST /sdk/v1/token-savings endpoint. Select any of seven token-saving methods and receive the final text plus savings estimates. Its deterministic output helps keep repeated prompt content hash stable and cache friendly when you apply it consistently.

SDK endpointUse it toInput
POST /sdk/v1/checkDetect, redact or block contentText, messages or JSON; inline images and documents on request checks
POST /sdk/v1/token-savingsOptimize text with explicit method selectionText only

To test a key before integrating it into your app, open the LLM Gateway Playground and select Quilr SDK, then Guardrail check or Token savings. For checks, choose Text, Messages, or JSON and select the check type and hashing mode. For token savings, enter text and select your methods. Both operations generate matching cURL, Python and JavaScript.

Common uses:

  • Check user input before forwarding to an LLM
  • Scan LLM responses before returning them to users
  • Scan file uploads, form fields, or other non-LLM content
  • Integrate with a self-hosted LiteLLM proxy
  • Optimize retrieved content, API results and tool output before sending them to your LLM

Authentication​

SDK mode requires a dedicated SDK key - regular LLM proxy keys are rejected with 403.

Create an app in Settings > AI Gateway > LLM Gateway > Create App, choose App-only credentials and select the Quilr SDK tile. It provides standalone content checks and token savings, has no models, and cannot be combined with other providers. Use the app's Quilr key as a Bearer token:

Authorization: Bearer sk-quilr-xxx

Api-Key: sk-quilr-xxx is also accepted.

You can optionally include an X-User-Email header for identity-aware enforcement if that is configured on the app.

Request Format​

POST /sdk/v1/check

Three input formats are supported. Send one content field: messages, text, or json.

Messages (conversation)​

Send an OpenAI-style conversation. Request checks select recent user messages for message-text detection according to your key settings. Attachment discovery also includes supported files in the supplied conversation history.

Set type: "request" or "response" explicitly. The explicit value wins over message roles. Without it, a final assistant message selects a response check and returns the text response shape; otherwise the messages are checked as a request.

{
"messages": [
{"role": "user", "content": "..."},
{"role": "assistant", "content": "..."}
],
"type": "request",
"metadata": {}
}

Text (raw string)​

Use this to check a single piece of text. Set type: "request" for input checks; omitting type checks raw text as a response.

{
"text": "some text to check",
"type": "request"
}

Images and documents (request checks)​

Set type: "request" and send the file bytes inline. For a conversation, put image and file blocks inside a message's content array. Text is optional: an image-only or document-only message is valid.

{
"type": "request",
"messages": [{
"role": "user",
"content": [
{"type": "text", "text": "Inspect these attachments."},
{
"type": "image_url",
"image_url": {"url": "data:image/png;base64,<BASE64_IMAGE_BYTES>"}
},
{
"type": "file",
"file": {
"filename": "sample.pdf",
"file_data": "data:application/pdf;base64,<BASE64_PDF_BYTES>"
}
}
]
}]
}

Replace the placeholders with standard base64 of each original file's bytes. Recognized image/file fields can also carry bare base64. Anthropic-style image and document blocks with a base64 source are accepted, along with the gateway's other supported inline content blocks.

The SDK uses the same attachment discovery as normal gateway requests. Data URLs embedded in message strings, text content parts, HTML and tool-result text are inspected too, including wrapped base64. For a single string, use request-side text:

{
"type": "request",
"text": "Inspect <img src=\"data:image/png;base64,<BASE64_IMAGE_BYTES>\">"
}

A bare base64 string in ordinary text is treated as text, not inferred to be an image. Remote URLs and provider file IDs are not fetched. The request still needs messages or text; a standalone attachment array is not a fourth input format.

Enable Scan attachments in the SDK app's Attachments settings, or the equivalent published Policy Engine control. OCR and document text use the enabled request-side text detections. Visual classification also needs Classify images and at least one enabled Image Risks type. See supported files and coverage.

This protection is request-side only. Response checks do not scan generated images or documents, and explicit json input remains a check of JSON values rather than an attachment envelope.

JSON (structured content)​

Put the value to scan in json. Pass an actual JSON value, rather than a serialized object inside text.

{
"type": "request",
"json": {
"customer": {
"firstName": "Alex",
"fullName": "Alex Morgan",
"name": "ALEX"
},
"contacts": [{"email": "alex@example.com"}],
"active": true,
"notes": null
},
"hashing_mode": "case_insensitive",
"metadata": {"source": "customer-profile"}
}

json accepts objects, arrays, and scalar roots, including strings, numbers, booleans, and null. JSON checks default to type: "request"; use "response" to apply response-side scopes. JSON-looking content in text stays in text mode.

String and number values are scanned together as one document. Field and parent names provide context, including readable camelCase and snake_case labels. Keys are preserved and entity detections found only in keys are discarded. Key names do not force detection: your configured categories, sensitivities, scopes, and actions still apply. For example, the SDK's labeled-name fallback requires the NAME subcategory sensitivity to be high.

A detected entity can match occurrences in multiple values, including under different keys. This is document-wide entity detection, not independent classification of each field.

Request parameters​

FieldTypeBehavior
messagesArray of messagesConversation input, including supported inline image/file blocks and embedded data URLs on request checks. Use one primary content field per request.
textStringRaw text input. With type: "request", supported embedded data URLs also reach attachment scanning.
jsonAny JSON valueStructured input. Cannot be combined with text or messages, even if those fields are null.
type"request" or "response"Explicit direction takes precedence over message roles. Without it, JSON defaults to request, raw text defaults to response, and messages ending in an assistant message use response checks.
hashing_mode"case_sensitive" or "case_insensitive"Optional; defaults to "case_sensitive". Controls placeholder hashes in every input format.
metadataJSON metadataOptional caller metadata stored with the SDK check.

Response​

Completed checks return HTTP 200, including content blocked by guardrails. Authentication, validation, and policy-access errors use non-2xx responses. The response shape depends on the input and check type. Use action for application control flow. Message-text and JSON findings appear in predictions; attachment findings and scan coverage appear in attachments.

Messages response​

{
"status": "safe | redacted | blocked",
"action": "allow | redact | block",
"messages": [...],
"blocked_text": "...",
"predictions": [
{
"id": "...",
"name": "...",
"type": "redact",
"sensitive_entities": ["123-45-6789"],
"entity_texts_with_subcategories": {
"123-45-6789": "SOCIAL SECURITY NUMBER"
}
}
],
"categories_detected": ["pii", "email", "ssn"],
"similar_entities": [],
"similar_entities_truncated": false,
"placeholder_masking": {
"hashing_mode": "case_sensitive",
"text": "My SSN is <qe-01a54629efb95228>.",
"messages": [
{ "role": "user", "content": "My SSN is <qe-01a54629efb95228>." }
],
"placeholders": [
{
"placeholder": "<qe-01a54629efb95228>",
"value": "123-45-6789",
"sub_category": "SOCIAL SECURITY NUMBER",
"category_id": "data_risk_category_pii",
"action": "redact",
"message_index": 0
}
]
},
"error": {...}
}
  • messages - the (possibly redacted) messages array; null if blocked
  • blocked_text - only present when status is blocked
  • predictions - rule-level details, including exact sensitive_entities and entity_texts_with_subcategories
  • error - only present when status is blocked

Text response​

{
"status": "safe | redacted | blocked",
"action": "allow | redact | block",
"original_text": "...",
"processed_text": "...",
"predictions": [
{
"id": "...",
"name": "...",
"type": "redact",
"sensitive_entities": ["555-867-5309"],
"entity_texts_with_subcategories": {
"555-867-5309": "PHONE NUMBER"
}
}
],
"categories_detected": ["pii", "phone"],
"similar_entities": [],
"similar_entities_truncated": false,
"placeholder_masking": {
"hashing_mode": "case_sensitive",
"text": "Call me at <qe-59c0b4a6fc3c3b2c>.",
"messages": null,
"placeholders": [
{
"placeholder": "<qe-59c0b4a6fc3c3b2c>",
"value": "555-867-5309",
"sub_category": "PHONE NUMBER",
"category_id": "data_risk_category_pii",
"action": "redact"
}
]
},
"error": {...}
}
  • processed_text - the redacted text; null if blocked
  • predictions - rule-level details, including exact sensitive_entities and entity_texts_with_subcategories
  • error - only present when status is blocked

Attachment results​

When request attachments are recognized, the response adds attachments with scanned, summary and items. This describes file coverage and findings separately from message-text predictions. It contains metadata, not the file bytes or extracted text. The paths below are relative to attachments.

FieldHow to use it
scannedfalse when attachment scanning was disabled. true means the attachment pipeline ran; inspect each item's coverage too.
summary.incomplete_countNumber of files whose requested scan could not finish. An allowed request can still have incomplete coverage.
items[].id, items[].path, items[].kindIdentify the attachment and its location in your input.
items[].status, items[].reason, items[].stages_run, items[].ocr_usedDescribe whether the file was scanned, skipped, unreadable or incomplete, and which stages ran.
items[].data_types, items[].image_labelsThe detected attachment text types and visual image categories.
items[].action, items[].stripped, items[].incomplete, items[].unscannableThe per-file action and coverage flags.

Monitor keeps the original inline content. Redact, Partial redact or Tokenize removes the whole affected attachment, rather than editing its pixels or document contents. The returned messages, processed_text and applicable placeholder view reflect that removal. An attachment-only removal still produces status: "redacted", action: "redact", even if the surrounding message text is unchanged.

Attachment blocks, including file type and configured limit blocks, use the existing HTTP 200 SDK verdict: status: "blocked", action: "block" and an error with the attachment code. Do not forward content after a block. Attachment errors do not appear as text entity placeholders.

The SDK also recognizes the gateway's supported media in request extension fields. When an attachment is outside the primary messages or text field, the response adds processed_body: the complete sanitized input body, with primary text redactions applied too. It is null on a block. Use that body to retain rewritten extension fields; replacing only messages would not apply a removal elsewhere in the input.

Check coverage, not just HTTP 200

Scanner errors, timeouts and unfinished scans fail open. They can return action: "allow" while the attachment metadata reports incomplete coverage. An unreadable-file policy is a separate control. Check the per-file status and flags, and see Actions and unreadable files.

JSON response​

This example assumes NAME is detected and its configured action is redact:

{
"status": "redacted",
"action": "redact",
"original_json": {"name": "alex", "active": true, "notes": null},
"processed_json": {"name": "XXXXXXXX", "active": true, "notes": null},
"original_text": "{\"name\": \"alex\", \"active\": true, \"notes\": null}",
"processed_text": "{\"name\": \"XXXXXXXX\", \"active\": true, \"notes\": null}",
"predictions": [
{
"id": "data_risk_category_pii",
"type": "redact",
"sensitive_entities": ["alex"],
"entity_texts_with_subcategories": {"alex": "NAME"},
"entity_json_paths": {"alex": ["/name"]}
}
],
"categories_detected": ["pii"],
"placeholder_masking": {
"hashing_mode": "case_sensitive",
"json": {"name": "<qe-4135aa9dc1b842a6>", "active": true, "notes": null},
"text": "{\"name\": \"<qe-4135aa9dc1b842a6>\", \"active\": true, \"notes\": null}",
"messages": null,
"placeholders": [
{
"placeholder": "<qe-4135aa9dc1b842a6>",
"value": "alex",
"sub_category": "NAME",
"category_id": "data_risk_category_pii",
"action": "redact",
"json_paths": ["/name"]
}
]
},
"similar_entities": [],
"similar_entities_truncated": false
}
  • original_json, processed_json, and placeholder_masking.json are actual JSON values. The corresponding *_text fields contain serialized JSON for compatibility.
  • processed_json follows the configured redact or partial-redact action. SDK Mode has no response to restore, so a Tokenize action is applied as redact. Safe and monitor-only values stay unchanged. Keys, nesting, array order, empty containers, booleans, and null are preserved. A masked number becomes a string; unchanged numbers keep their numeric type.
  • Blocked results have action: "block", processed_json: null, processed_text: null, and error details. Placeholder output remains available for inspection and does not override the block. Check action before consuming content; null can also be a valid safe JSON root.
  • entity_json_paths maps each detected entity to matching value locations. Placeholder entries include json_paths for replaced locations. These are JSON Pointers: /contacts/0/email addresses an array item, ~ is escaped as ~0, / as ~1, and "" means the root.

Placeholder masking and hashing mode​

Every completed check includes placeholder_masking, an additional view with full placeholders for entities whose effective action is redact, partial-redact, or block. Monitor-only values remain unchanged. Use placeholder_masking.messages for request messages, .text for raw text, or .json for JSON input. Text and JSON checks set .messages to null.

Tokens have the format <qe-{16-char-hash}>. The same source value and hashing mode produce the same placeholder, so the identifier for that value stays stable. placeholders[] retains original values and detection metadata so your application can restore values using the mapping.

hashing_modeHash inputExample
case_sensitive (default)Exact matched source valueAlex and alex have different hashes.
case_insensitiveSource value after Unicode case foldingAlex, alex, and ALEX all produce <qe-4135aa9dc1b842a6>.

The selected mode is echoed in placeholder_masking.hashing_mode. It applies to all placeholder entity types and changes hashing only; detection, ordinary redaction, and policy actions are unaffected. Whitespace, punctuation, and accents remain part of the hash input.

Repeated identical JSON values share a placeholder entry with every replaced path. In case-insensitive mode, different source spellings can share a token but retain separate mapping entries with their exact value and json_paths or message_index. Preserve the location and source spelling if you need to restore exact capitalization: the shared token alone cannot distinguish it.

Similar entities​

Every completed check also includes similar_entities and similar_entities_truncated. These are response fields, not request parameters. When both names in {"first": "Alex", "full": "Alex Morgan"} are detected, a case-insensitive check can return:

{
"similar_entities": [
{
"relationship": "possible_name_match",
"reason": "name_contains",
"entities": [
{
"value": "Alex",
"hash": "4135aa9dc1b842a6",
"placeholder": "<qe-4135aa9dc1b842a6>",
"category_id": "data_risk_category_pii",
"sub_category": "NAME",
"json_paths": ["/first", "/full"]
},
{
"value": "Alex Morgan",
"hash": "60705d930a9ed84f",
"placeholder": "<qe-60705d930a9ed84f>",
"category_id": "data_risk_category_pii",
"sub_category": "NAME",
"json_paths": ["/full"]
}
]
}
],
"similar_entities_truncated": false
}

Pairs are advisory textual similarities, not verified identity matches, and do not merge hashes or change detection or masking. Only detected person-name subcategories within the same category are compared; passwords, identifiers, emails, and usernames are excluded. Names that were not detected are not inferred.

reasonMeaning
case_variantSame value after case folding.
normalized_name_matchSame name tokens after Unicode, spacing, and punctuation normalization.
name_containsOne detected name is a contiguous sequence of whole name tokens in the other, including a surname.

Matching does not use fuzzy spelling, initials, or partial-word matches such as Ann/Anna. Pairs do not imply transitive identity groups. Similarity normalization does not change the hash input: hashes always follow hashing_mode, and short and full names keep separate hashes.

Entities retain actual source spellings and json_paths for JSON or message_indices for messages. JSON keys are excluded. Monitor-only or overlapping name detections may appear, so a reported hash need not appear in the masked output. With no matching names the list is [].

Candidate values and returned pairs are capped at 256 each. similar_entities_truncated: true signals either limit; this limit affects only these advisory results, not detection or masking.

Validation errors​

Check the HTTP status before reading a check result. Validation errors use an error object with message, type: "invalid_request_error", and code.

HTTP statuserror.codeCause
400invalid_jsonMalformed request JSON or a request envelope that is not an object.
400missing_contentMissing or invalid text/messages content.
400invalid_json_inputjson combined with another content field, invalid JSON-mode type, invalid Unicode, NaN/Infinity, or nesting beyond 64 levels.
400invalid_hashing_modeAny hashing mode other than the two supported strings, including null or booleans.

Code Examples​

The examples use US East (guardrails-usa-2.quilr.ai); replace it with your region's host from the integration guide.

Image and PDF request check - Python​

This sends local file bytes directly to the SDK check. It does not make a model call. Configure the SDK app's attachment protections and data types before testing.

import base64
import os
from pathlib import Path

import requests

def file_data_url(path, mime):
encoded = base64.b64encode(Path(path).read_bytes()).decode("ascii")
return f"data:{mime};base64,{encoded}"

response = requests.post(
"https://guardrails-usa-2.quilr.ai/sdk/v1/check",
headers={"Authorization": f"Bearer {os.environ['QUILR_SDK_API_KEY']}"},
json={
"type": "request",
"messages": [{
"role": "user",
"content": [
{"type": "image_url", "image_url": {
"url": file_data_url("sample.png", "image/png"),
}},
{"type": "file", "file": {
"filename": "sample.pdf",
"file_data": file_data_url("sample.pdf", "application/pdf"),
}},
],
}],
},
timeout=60,
)
response.raise_for_status()
check = response.json()
print(check.get("attachments"))
if check["action"] == "block":
raise RuntimeError(check["error"]["message"])
checked_messages = check["messages"] # Use these, not the original messages.

For a proxied model request with the same image/file blocks, see Send images and documents.

Structured JSON - Python​

Send the JSON value without stringifying it. Check the action first and read processed_json directly, including when the safe result is null, false, 0, or an empty value.

import requests

response = requests.post(
"https://guardrails-usa-2.quilr.ai/sdk/v1/check",
headers={"Authorization": "Bearer sk-quilr-xxx"},
json={
"type": "request",
"json": {"name": "Alex", "active": True, "notes": None},
"hashing_mode": "case_insensitive",
"metadata": {"source": "customer-profile"},
},
timeout=15,
)
response.raise_for_status()
result = response.json()
if result["action"] == "block":
raise ValueError(f"Blocked: {result['categories_detected']}")
safe_json = result["processed_json"]
placeholder_json = result["placeholder_masking"]["json"]
possible_name_matches = result["similar_entities"]

Structured JSON - JavaScript​

const response = await fetch("https://guardrails-usa-2.quilr.ai/sdk/v1/check", {
method: "POST",
headers: {
Authorization: "Bearer sk-quilr-xxx",
"Content-Type": "application/json",
},
body: JSON.stringify({
type: "request",
json: { name: "Alex", active: true, notes: null },
hashing_mode: "case_insensitive",
}),
});
if (!response.ok) throw new Error(`Check failed: ${response.status}`);
const result = await response.json();
if (result.action === "block") throw new Error("Content blocked");
const safeJson = result.processed_json;
const placeholderJson = result.placeholder_masking.json;
const possibleNameMatches = result.similar_entities;

Structured JSON - cURL​

curl -X POST https://guardrails-usa-2.quilr.ai/sdk/v1/check \
-H "Authorization: Bearer sk-quilr-xxx" \
-H "Content-Type: application/json" \
-d '{
"type": "response",
"json": {"name": "Alex", "active": true, "notes": null},
"hashing_mode": "case_insensitive"
}'

Python - httpx (async)​

A typical pattern: check the user message before sending it to your LLM, then check the LLM response before returning it to the user.

import httpx

QUILR_BASE = "https://guardrails-usa-2.quilr.ai"
QUILR_SDK_KEY = "sk-quilr-xxx"

async def check_messages(messages: list[dict]) -> dict:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{QUILR_BASE}/sdk/v1/check",
headers={"Authorization": f"Bearer {QUILR_SDK_KEY}"},
json={"messages": messages, "type": "request"},
timeout=5,
)
resp.raise_for_status()
return resp.json()

async def check_text(text: str, type_: str = "response") -> dict:
async with httpx.AsyncClient() as client:
resp = await client.post(
f"{QUILR_BASE}/sdk/v1/check",
headers={"Authorization": f"Bearer {QUILR_SDK_KEY}"},
json={"text": text, "type": type_},
timeout=5,
)
resp.raise_for_status()
return resp.json()

# --- Usage ---

import asyncio
from openai import AsyncOpenAI

openai = AsyncOpenAI(api_key="sk-openai-xxx")

async def safe_chat(user_message: str) -> str:
messages = [{"role": "user", "content": user_message}]

# 1. Check input
result = await check_messages(messages)
if result["status"] == "blocked":
raise ValueError(f"Input blocked: {result['categories_detected']}")
if result["status"] == "redacted":
messages = result["messages"] # use redacted version

# 2. Call LLM
response = await openai.chat.completions.create(
model="gpt-4o-mini",
messages=messages,
)
reply = response.choices[0].message.content

# 3. Check output
result = await check_text(reply, type_="response")
if result["status"] == "blocked":
raise ValueError(f"Response blocked: {result['categories_detected']}")
if result["status"] == "redacted":
reply = result["processed_text"]

return reply

asyncio.run(safe_chat("What is my SSN?"))

Python - requests (sync)​

import requests

QUILR_BASE = "https://guardrails-usa-2.quilr.ai"
QUILR_SDK_KEY = "sk-quilr-xxx"

def check_text(text: str, type_: str = "response") -> dict:
resp = requests.post(
f"{QUILR_BASE}/sdk/v1/check",
headers={"Authorization": f"Bearer {QUILR_SDK_KEY}"},
json={"text": text, "type": type_},
timeout=5,
)
resp.raise_for_status()
return resp.json()

# Check a piece of text before storing or displaying it
result = check_text("My credit card is 4111 1111 1111 1111", type_="request")

match result["status"]:
case "safe":
print("No issues found")
case "redacted":
print("Cleaned text:", result["processed_text"])
case "blocked":
print("Blocked. Detected:", result["categories_detected"])

JavaScript / TypeScript - fetch​

const QUILR_BASE = "https://guardrails-usa-2.quilr.ai";
const QUILR_SDK_KEY = "sk-quilr-xxx";

async function checkMessages(messages: Array<{ role: string; content: string }>) {
const res = await fetch(`${QUILR_BASE}/sdk/v1/check`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${QUILR_SDK_KEY}`,
},
body: JSON.stringify({ messages, type: "request" }),
signal: AbortSignal.timeout(5000),
});
if (!res.ok) throw new Error(`Quilr error: ${res.status}`);
return res.json();
}

async function checkText(text: string, type: "request" | "response" = "response") {
const res = await fetch(`${QUILR_BASE}/sdk/v1/check`, {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${QUILR_SDK_KEY}`,
},
body: JSON.stringify({ text, type }),
signal: AbortSignal.timeout(5000),
});
if (!res.ok) throw new Error(`Quilr error: ${res.status}`);
return res.json();
}

// Example: guard a chat endpoint
async function safeChat(userMessage: string): Promise<string> {
const messages = [{ role: "user", content: userMessage }];

const inputResult = await checkMessages(messages);
if (inputResult.status === "blocked") {
throw new Error(`Blocked: ${inputResult.categories_detected.join(", ")}`);
}
const checkedMessages =
inputResult.status === "redacted" ? inputResult.messages : messages;

// ... call your LLM with checkedMessages ...
const llmReply = "...";

const outputResult = await checkText(llmReply, "response");
if (outputResult.status === "blocked") {
throw new Error(`Response blocked: ${outputResult.categories_detected.join(", ")}`);
}
return outputResult.status === "redacted" ? outputResult.processed_text : llmReply;
}

cURL​

# Check raw text
curl -X POST https://guardrails-usa-2.quilr.ai/sdk/v1/check \
-H "Authorization: Bearer sk-quilr-xxx" \
-H "Content-Type: application/json" \
-d '{"text": "Call me at 555-867-5309", "type": "request"}'

# Check a conversation
curl -X POST https://guardrails-usa-2.quilr.ai/sdk/v1/check \
-H "Authorization: Bearer sk-quilr-xxx" \
-H "Content-Type: application/json" \
-d '{
"messages": [
{"role": "user", "content": "What is the password for admin@acme.com?"}
],
"type": "request"
}'

TrueFoundry Integration​

Use QuilrAI directly as a TrueFoundry custom guardrail for LLM inputs and outputs. Configure Mutate, Custom Bearer Auth with a quilr_sdk key, and the dedicated POST /sdk/v1/check/truefoundry endpoint. TrueFoundry sends its own request/response envelope and consumes the returned verdict, transformed, and result fields.

See TrueFoundry Integration for dashboard setup, selecting input/output rails, complete request examples, streaming requirements, and error handling.

LiteLLM Guardrails Integration​

If you run a self-hosted LiteLLM proxy, you can add QuilrAI as a native LiteLLM guardrail. The plugin calls /sdk/v1/check for you on requests, responses, tool calls and MCP tool calls. No changes are needed in your application code.

At a glance (QuilrUnifiedGuardrail):

  • Supported: prompts and answers on OpenAI Chat Completions, Anthropic Messages, the Responses API, text completion and Gemini; embeddings; image generation prompts; text to speech; rerank queries; tool calls; MCP tool calls (all tools, or only the ones you choose); streaming; guardrail records in Langfuse and other LiteLLM logging integrations; fail-open or fail-closed
  • Not checked: images, tool definitions sent to the model, and transcribed audio text (a LiteLLM limit)

Details are in What is supported and Limits.

Which class to use​

The plugin file has two classes. Use QuilrUnifiedGuardrail for new setups.

ClassCoversUse it when
QuilrUnifiedGuardrail (recommended)All the API formats listed below, tool calls, MCP tool calls, streaming, logging records, fail-closedNew setups, or anything beyond plain chat
QuilrGuardrailRequest and response text of OpenAI Chat Completions and the Responses APIExisting setups that already use it. It is kept unchanged

QuilrUnifiedGuardrail uses LiteLLM's unified guardrail interface, so LiteLLM extracts the text from each API format and the plugin checks it. Tested with LiteLLM 1.106.2.

Prerequisites​

  • A Quilr guardrails API key (a quilr_sdk key), from either Quilr-hosted guardrails or a self-hosted Quilr guardrails deployment
  • Download quilr_litellm_guardrails.py and place it in the same directory as your LiteLLM config.yaml
note

LiteLLM resolves custom guardrails by module path relative to the proxy working directory, so the plugin is provided as a single file placed alongside config.yaml. There is no package to install.

Environment variables​

QUILR_GUARDRAILS_KEY=sk-quilr-XXXXXXXXX
QUILR_GUARDRAILS_BASE_URL=<QUILR GUARDRAILS BASE URL>
QUILR_GUARDRAILS_TIMEOUT=3 # Optional: seconds before a check times out (default: 3)
QUILR_GUARDRAILS_FAIL_CLOSED=false # Optional: block when Quilr can't be reached (QuilrUnifiedGuardrail only)
QUILR_GUARDRAILS_SCAN_MCP_TOOLS=all # Optional: which MCP tools to check (QuilrUnifiedGuardrail only)
VariableRequiredDefaultDescription
QUILR_GUARDRAILS_KEYYes-Your quilr_sdk API key
QUILR_GUARDRAILS_BASE_URLNohttps://guardrails.quilr.aiOverride with the closest regional endpoint for production or with a self-hosted deployment URL
QUILR_GUARDRAILS_TIMEOUTNo3Seconds before a check times out
QUILR_GUARDRAILS_FAIL_CLOSEDNofalsetrue blocks requests when Quilr can't be reached (QuilrUnifiedGuardrail only). See When Quilr can't be reached
QUILR_GUARDRAILS_SCAN_MCP_TOOLSNoallWhich MCP tools to check: all, none, or a comma-separated mix of readonly, writeonly, destructive and tool names (QuilrUnifiedGuardrail only). See Choosing which MCP tools to check
APPLY_QUILR_GUARDRAILS_FOR_MODELSNo(all)Comma-separated list of models to restrict guardrails to
APPLY_QUILR_GUARDRAILS_FOR_KEY_NAMESNo(all)Comma-separated list of LiteLLM key names to restrict guardrails to

What is supported​

All of this applies to QuilrUnifiedGuardrail.

Modes​

Set the mode per guardrail in config.yaml. You can add the same class several times with different modes.

ModeWhen it runsWhat it checksCan blockCan redact
pre_callBefore the LLM callPrompt, chat history, tool-call arguments in the historyYesYes
during_callIn parallel with the LLM callPromptYesNo
post_callAfter the LLM callAnswer, tool calls in the answer, streamed answersYesYes
logging_onlyAfter the call, on copiesPrompt and answerNo (records the result only)No
pre_mcp_callBefore an MCP tool runs, and when clients list toolsTool arguments, tool descriptionsYesYes (arguments)
during_mcp_callIn parallel with the MCP toolTool argumentsYesNo
post_mcp_callAfter the MCP tool runs, before the result reaches the modelTool resultYesYes

pre_call or during_call? during_call adds no latency, but the LLM has already received the prompt when the check finishes, so it can only block, not redact. Use pre_call to stop data before it reaches the LLM.

API formats and endpoints​

EndpointWhat is checked
OpenAI Chat Completions (/v1/chat/completions)Prompt, answer, tool calls, streaming
Anthropic Messages (/v1/messages)Prompt, answer, streaming (block only), tool calls (block only)
OpenAI Responses API (/v1/responses)Input and output
Text completion (/v1/completions)Prompt and answer
Gemini (generateContent)Prompt and answer
Embeddings (/v1/embeddings)All inputs, checked together
Image generation (/v1/images/generations)Prompt
Text to speech (/v1/audio/speech)Input text
Rerank (/v1/rerank)Query (LiteLLM does not pass the documents to guardrails)
MCP (/mcp, /mcp-rest, tool search)Tool arguments, results and descriptions

What content is checked​

ContentHow it is checked
Prompts and answersAs text, or as chat messages when LiteLLM provides roles
Tool-call argumentsAll tool calls of a request or answer in one JSON check. Each argument value is checked, and redacted values are written back into the arguments
MCP tool arguments, results and descriptionsOne JSON check per call, so every value is checked
Several inputs without roles (for example an embeddings batch)One JSON check, so every input is checked
ImagesNot checked
Tool definitions sent to the model (tools)Not checked

Streaming​

With post_call, streamed answers are checked while they stream: every few chunks, and once at the end. A block ends the stream cleanly with the block message (finish_reason: content_filter in OpenAI format). Optional settings, under litellm_params:

SettingDefaultWhat it does
streaming_sampling_rate5Check the answer so far every Nth chunk. Each check is a Quilr call
streaming_end_of_stream_onlyfalsetrue: check once, after the whole answer has streamed (1 Quilr call, but the answer is already delivered)
streaming_buffer_until_moderatedfalsetrue: send nothing to the client until the check passes
streaming_buffer_release_on_scanfalseWith buffering: release the held chunks after each passing check
streaming_transform_modeblock_onlyincremental_diff: also redact while streaming (OpenAI Chat Completions format only)
stream_holdback_chars64 with incremental_diffCharacters held back so a value split across chunks can still be redacted

Which setup to use:

  • Sensitive values must be redacted: streaming_transform_mode: incremental_diff
  • Block only, and nothing may reach the client before the check: streaming_buffer_until_moderated: true
  • Fewest Quilr calls (monitoring): streaming_end_of_stream_only: true, or a higher streaming_sampling_rate
note

Only incremental_diff redacts while streaming. In the other setups, if Quilr redacts, LiteLLM drops the redaction and the client gets the original text. Use them to block.

Logging and tracing​

Each check adds a guardrail record to LiteLLM's logs, so it shows in Langfuse, Datadog and the other LiteLLM logging integrations:

  • guardrail_provider: quilr
  • guardrail_status: success, guardrail_intervened (blocked), guardrail_flagged (categories found, allowed), guardrail_failed_to_respond (Quilr unreachable) or not_run (MCP tool skipped by QUILR_GUARDRAILS_SCAN_MCP_TOOLS)
  • guardrail_response: Quilr's status, action and categories
  • masked_entity_count: how many checks redacted each category

The record never contains the checked content.

To group checks by conversation and user in Quilr, the plugin sends:

  • X-Conversation-Id: the request's metadata.session_id (or LiteLLM's session ID)
  • X-User-Email: the first value that looks like an email from metadata.trace_user_id, the request's user, the end user, or the LiteLLM key owner's email. If Quilr rejects the email (domain not allowed on the app), the check is retried without it
  • The session and trace IDs in the check's metadata, to match Quilr logs with LiteLLM or Langfuse traces

When Quilr can't be reached​

By default the plugin fails open: if Quilr times out or returns an error, the content passes unchecked, a warning is logged, and the logging record shows guardrail_failed_to_respond. To block instead, set quilr_fail_closed: true on each guardrail that should fail closed, or QUILR_GUARDRAILS_FAIL_CLOSED=true for all of them. Blocked requests return HTTP 503.

If you use fail-closed, consider a higher QUILR_GUARDRAILS_TIMEOUT so slow responses aren't blocked.

Choosing which MCP tools to check​

By default every MCP tool is checked. To check only some, set QUILR_GUARDRAILS_SCAN_MCP_TOOLS for all guardrails, or quilr_scan_mcp_tools under one guardrail's litellm_params.

ValueTools checked
all (default, also when not set)Every tool
noneNo MCP tools
readonlyTools labeled read-only (MCP readOnlyHint: true)
writeonlyTools not labeled read-only
destructiveTools that may delete or overwrite data (not read-only, and MCP destructiveHint not false)
run_sql,read_logsThese tools only
writeonly,run_sqlA mix: a tool is checked if any item matches

For example, with tools read_logs (read-only), run_sql (read-only), delete_file (destructive), send_email (destructiveHint: false) and get_time (no labels):

SettingCheckedSkipped
writeonlydelete_file, send_email, get_timeread_logs, run_sql
destructivedelete_file, get_timeread_logs, run_sql, send_email
writeonly,run_sqldelete_file, send_email, get_time, run_sqlread_logs
run_sql,send_emailrun_sql, send_emailread_logs, delete_file, get_time
  • Applies to pre_mcp_call, during_mcp_call, post_mcp_call and the tool list check.
  • Give tool names as the MCP server names them (run_sql) or with LiteLLM's server prefix (myserver-run_sql). Names are case-sensitive. Label words are not, and can be written with - or _ (read-only, write_only). In config.yaml the value can also be a YAML list.
  • A tool without labels counts as writeonly and destructive (the MCP spec's defaults), so it is still checked by those.
  • The labels are read from the MCP server's tool list and cached for 5 minutes (a failed read is retried after 30 seconds). If the list can't be read, the tool counts as unlabeled.
  • MCP servers that need each caller's own login (a per-user token; OAuth servers are expected to behave the same): the plugin can't read their tool list, so their tools count as unlabeled. writeonly and destructive then check all of them, and readonly checks none. Use tool names for these servers. Servers whose login is set in the LiteLLM config work normally.
  • In during_mcp_call, LiteLLM doesn't pass the server name, so the labels are looked up on every MCP server. A tool name that exists on several servers counts as read-only only if it is read-only on all of them.
  • none together with other values is treated as all, with a warning in the log.
  • A skipped tool passes through unchanged. Its logging record has guardrail_status: not_run.
warning

The labels are set by the MCP server itself and are not verified. Read-only tools such as SQL or log readers often return the most sensitive data, so skip them only when you are sure.

LiteLLM config.yaml​

Input and output guardrails

guardrails:
- guardrail_name: "quilr-input"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "pre_call"
default_on: true

- guardrail_name: "quilr-output"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "post_call"
default_on: true

Input guardrail with lower latency (during_call, block only)

guardrails:
- guardrail_name: "quilr-input-duringcall"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "during_call"
default_on: true

Redaction while streaming

guardrails:
- guardrail_name: "quilr-output"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "post_call"
default_on: true
streaming_transform_mode: "incremental_diff"

MCP tool calls (LiteLLM as an MCP gateway)

guardrails:
- guardrail_name: "quilr-mcp-input"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "pre_mcp_call"
default_on: true

- guardrail_name: "quilr-mcp-output"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "post_mcp_call"
default_on: true

Block when Quilr can't be reached

guardrails:
- guardrail_name: "quilr-input"
litellm_params:
guardrail: quilr_litellm_guardrails.QuilrUnifiedGuardrail
mode: "pre_call"
default_on: true
quilr_fail_closed: true

For an existing QuilrGuardrail setup, use guardrail: quilr_litellm_guardrails.QuilrGuardrail with the pre_call, during_call or post_call modes.

Optional filtering​

You can limit which requests have guardrails applied using these environment variables:

# Only apply guardrails to specific models (comma-separated)
APPLY_QUILR_GUARDRAILS_FOR_MODELS=gpt-4,gpt-4o,claude-3-opus

# Only apply guardrails to specific API key names (comma-separated)
APPLY_QUILR_GUARDRAILS_FOR_KEY_NAMES=production-key,user-facing-key

If neither variable is set, guardrails apply to all requests. If both are set, a request must match both filters (AND logic) for guardrails to be applied.

note
  • LiteLLM gives MCP checks the model name mcp-tool-call, so APPLY_QUILR_GUARDRAILS_FOR_MODELS skips MCP checks unless you add mcp-tool-call to the list.
  • When APPLY_QUILR_GUARDRAILS_FOR_KEY_NAMES is set, requests on routes where LiteLLM does not pass the key name (for example /v1/messages) are not checked.

Enabling guardrails per request​

With default_on: true, guardrails run on every request. If you set default_on: false, pass the guardrail names in the request body instead:

curl -X POST http://localhost:4000/v1/chat/completions \
-H "Authorization: Bearer sk-litellm-xxx" \
-H "Content-Type: application/json" \
-d '{
"model": "gpt-4o-mini",
"messages": [{"role": "user", "content": "Hello"}],
"guardrails": ["quilr-input-duringcall", "quilr-output"]
}'

Behavior summary​

ModeWhat happens on blockedWhat happens on redacted
pre_callRequest rejected before the LLM is calledPrompt replaced with the redacted version before the LLM call
during_callRequest rejected, LLM response discardedNot applied (the LLM call is already in flight)
post_callResponse rejected; a stream ends with the block messageAnswer and tool-call arguments replaced with the redacted version (for streams, only with incremental_diff)
logging_onlyNot blocked, recorded in the logsNot applied, recorded in the logs
pre_mcp_callTool does not runTool runs with the redacted arguments
during_mcp_callTool call rejectedNot applied
post_mcp_callResult withheld from the modelModel receives the redacted result

Limits​

Not checked by the plugin:

  • Images in prompts (Quilr checks text only)
  • Tool definitions sent to the model (the tools list)

Limits that come from LiteLLM:

  • Redaction while streaming works only in the OpenAI Chat Completions format. Other formats can block a stream but not redact it.
  • Anthropic Messages (/v1/messages) answers: LiteLLM does not apply changed tool calls, so tool-call arguments can be blocked but not redacted.
  • Audio transcription (/v1/audio/transcriptions): LiteLLM's proxy does not run post-call guardrails on transcriptions, so the transcribed text is not checked.
  • Rerank: only the query is checked.
  • MCP tool labels (readonly, writeonly, destructive in QUILR_GUARDRAILS_SCAN_MCP_TOOLS): LiteLLM does not pass them to guardrails, so the plugin reads them from the server's tool list. For servers that need each caller's own login this isn't possible; use tool names there. See Choosing which MCP tools to check.