Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/pages/docs/command-center/concepts/api-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Agent Command Center returns standard HTTP error codes with structured JSON erro

### Guardrail blocked (403)

When a guardrail whose action is `block` triggers on a request, Agent Command Center returns 403. A pre-stage check returns it before the LLM is ever called; a post-stage check returns it instead of the model's response. A guardrail whose action is `warn` or `log` does not change the status: the call returns 200 with the header `x-agentcc-guardrail-triggered: true`.
When a pre-stage guardrail whose action is `block` triggers, Agent Command Center returns 403 before the LLM is ever called. A post-stage guardrail checks the model's response, and today a `block` from it does not return 403. On a non-streamed response, a guardrail set up in the dashboard lets the response through with status 200 and the header `x-agentcc-guardrail-triggered: true`, and a synchronous `config.yaml` rule makes `/v1/chat/completions` return a 500 `internal_error` ("no response from provider") instead of the response. A guardrail whose action is `warn` or `log` does not change the status: the call returns 200 with the header `x-agentcc-guardrail-triggered: true`. On a streamed response the status and headers are sent when the stream starts, so only a pre-stage check can return 403 or add the header; see [Streaming Guardrails](/docs/command-center/features/guardrails#streaming-guardrails).

```json
{
Expand Down
76 changes: 60 additions & 16 deletions src/pages/docs/command-center/features/guardrails.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: "Enforce PII detection, prompt injection blocking, and content mode

## About

Guardrails are safety checks that run on every request and response flowing through Agent Command Center. They catch dangerous or unwanted content before it reaches the LLM (pre-processing) or before it reaches your users (post-processing).
Guardrails are safety checks that run on every request and response flowing through Agent Command Center. They catch dangerous or unwanted content before it reaches the LLM (pre-processing) or before it reaches your users (post-processing; for streamed responses, see [Streaming Guardrails](#streaming-guardrails)).

<Warning>
When [caching](/docs/command-center/features/caching) is on and a request is served from an exact match hit, guardrails that check the request (Pre stage) do not run on that request. Guardrails that check the response (Post stage) still run on the cached response before it is returned. Disable caching for traffic that must be screened on every request.
Expand All @@ -25,26 +25,26 @@ When [caching](/docs/command-center/features/caching) is on and a request is ser

## Built-in Guardrail Types

Agent Command Center includes 16 guardrail types covering common safety scenarios.
Agent Command Center includes 16 guardrail types covering common safety scenarios. The Stage column is the stage each type runs at when you set it up in the dashboard; today the gateway uses it whatever stage the guardrail is set to. A rule in `config.yaml` runs at the `stage` the rule sets.

| Guardrail Type | Stage | What it detects |
|---|---|---|
| PII Detection | Pre | Emails, SSNs, credit cards, phone numbers, addresses |
| Prompt Injection | Pre | Attempts to override system prompts or extract instructions |
| Content Moderation | Pre/Post | Hate speech, threats, sexual content, violence |
| Content Moderation | Pre | Hate speech, threats, sexual content, violence |
| Secret Detection | Pre | API keys, passwords, tokens, credentials |
| Hallucination Detection | Post | Factually incorrect or fabricated information |
| Topic Restriction | Pre | Blocks requests on restricted topics |
| Language Detection | Pre | Enforces allowed languages |
| Data Leakage Prevention | Pre/Post | Prevents sensitive data from being processed |
| Blocklist | Pre/Post | Custom word/phrase blocklists |
| Data Leakage Prevention | Post | Prevents sensitive data from being processed |
| Blocklist | Pre | Custom word/phrase blocklists |
| System Prompt Protection | Pre | Prevents system prompt extraction attempts |
| Tool Permissions | Pre | Validates tool/function call permissions |
| Input Validation | Pre | Validates input format and structure |
| MCP Security | Pre | Validates MCP protocol security |
| Custom Expression Rules | Pre/Post | Custom logic via expressions |
| Webhook (BYOG) | Pre/Post | Custom guardrails via webhook |
| Future AGI Evaluation | Post | Future AGI's proprietary evaluation models |
| Custom Expression Rules | Pre | Custom logic via expressions |
| Webhook (BYOG) | Pre | Custom guardrails via webhook |
| Future AGI Evaluation | Pre | Future AGI's proprietary evaluation models |

---

Expand Down Expand Up @@ -73,7 +73,7 @@ Choose how Agent Command Center handles guardrail violations.

| Mode | HTTP Status | Behavior |
|---|---|---|
| Enforce | 403 | Request blocked, error returned to client |
| Enforce | 403 (pre-stage checks) | Request blocked, error returned to client. A block from a post-stage check does not return 403 today; see [Guardrail blocked (403)](/docs/command-center/concepts/api-reference#guardrail-blocked-403) |
| Monitor | 200 | Request proceeds, warning logged |
| Log | 200 | Request proceeds, violation logged silently |

Expand Down Expand Up @@ -421,11 +421,18 @@ Use Redact or Mask to sanitize sensitive data while allowing the request to proc

## Streaming Guardrails

Guardrails work with streaming responses. Pre-processing guardrails run before streaming begins. Post-processing guardrails accumulate the full streamed response before evaluation.
When you stream a chat completion (`"stream": true` on `/v1/chat/completions`), Agent Command Center sends the response headers first, then the response in chunks as the model generates it. A guardrail's stage decides what it can still do:

- **Sync + block**: The stream terminates immediately if a violation is detected
- **Sync + warn**: A warning header is added, the stream continues
- **Async**: The guardrail runs fire-and-forget in the background: the stream is never interrupted
| Stage | When it runs on a stream | Effect |
|---|---|---|
| Pre (checks the request) | Before the stream opens | A `block` returns the 403 `content_blocked` error instead of a stream. A `warn` or `log` adds `x-agentcc-guardrail-triggered: true` to the stream's headers |
| Post (checks the response) | After the last chunk of response text, before `data: [DONE]` | Cannot stop or take back anything, because the headers and the response text have already gone out. `data: [DONE]` waits until these checks finish or time out. The result is recorded in the request log |

The table applies to guardrails set up in the dashboard and to `config.yaml` rules, except a `config.yaml` rule with `mode: "async"`: it runs in the background, so it cannot block the request, add the header, or delay `data: [DONE]`, and it reports a failure only in the gateway's own log output. Guardrails set up in the dashboard or through the SDK always run synchronously, whatever `mode` they carry.

Post-stage guardrails see the streamed text only on `/v1/chat/completions` and `/v1/completions`, and on a self-hosted gateway only when it records request and response bodies (`logging.request_logging.include_bodies: true`, or `otel.include_bodies: true` with `otel.enabled: true`). Otherwise they have no response text to check.

To act on a response while it streams, run a self-hosted gateway and [check the stream as it is generated](#check-the-stream-as-it-is-generated).

<CodeGroup>
```python Python
Expand All @@ -437,13 +444,13 @@ client = AgentCC(
control_plane_url="https://api.futureagi.com",
)

# Streaming with guardrails active on this key/org
# A pre-stage block raises here, before any chunk arrives
for chunk in client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": "Tell me about security."}],
stream=True,
):
if chunk.choices[0].delta.content:
if chunk.choices and chunk.choices[0].delta.content:
print(chunk.choices[0].delta.content, end="", flush=True)
```

Expand All @@ -455,6 +462,7 @@ const client = new AgentCC({
baseUrl: "https://gateway.futureagi.com",
});

// A pre-stage block throws here, before any chunk arrives
const stream = await client.chat.completions.create({
model: "gpt-4o-mini",
messages: [{ role: "user", content: "Tell me about security." }],
Expand All @@ -468,8 +476,44 @@ for await (const chunk of stream) {
```
</CodeGroup>

### Check the stream as it is generated

On a self-hosted gateway, the post-stage rules in [`config.yaml`](/docs/command-center/deployment/self-hosted#configuration-file) (`guardrails.rules`) can check a `/v1/chat/completions` response while the model is still generating it. This is off by default. Turn it on with `guardrails.streaming`:

```yaml
guardrails:
enabled: true
rules:
- name: "content-moderation"
stage: "post"
mode: "sync"
action: "block"
threshold: 0.7
streaming:
enabled: true
check_interval: 100 # Bytes of new response text between checks (default: 100)
failure_action: "stop" # "stop" (default) or "disclaimer"
```

Each time another `check_interval` bytes of text have arrived, the gateway holds the chunk that completes the interval and runs these rules on the whole response so far. It runs them once more when the model finishes, if text has arrived since the last check. Chunks in between are sent without waiting for a check, so up to `check_interval` bytes of text can reach the client before a check catches it, and a response shorter than `check_interval` is checked only after all of it has been sent. The stream waits while a check runs, so a smaller interval catches content sooner but runs more checks. Only `sync` rules whose action is `block` affect the stream:

| `failure_action` | When a check fails |
|---|---|
| `stop` (default) | If a check during the stream fails, the gateway sends an error event instead of the chunk it held and ends the stream without `data: [DONE]`. If the check at the end fails, the whole response has already been sent: the error event follows it, and the stream still ends with `data: [DONE]` |
| `disclaimer` | The stream continues. If the check at the end fails, the gateway sends one more content chunk after the one with `finish_reason`: two newlines followed by `[Content warning: <guardrail message>]`. A check that fails during the stream adds no notice |

With `disclaimer`, the check at the end runs only if text arrived after the last interval check. If the last chunk of text completes an interval, the response gets no notice, even when that check fails.

The error event that `stop` sends:

```text
data: {"error":{"message":"Content moderation triggered: violence (score: 1.00)","type":"guardrail_error","param":null,"code":"stream_blocked"}}
```

The HTTP status stays 200, because it was sent with the headers, and the chunks sent before the check are not taken back.

<Note>
Post-processing guardrails (stage: "post") accumulate the complete streamed response before evaluation. If a violation is detected in sync+block mode, the stream terminates and the client receives an error. Any chunks already delivered cannot be recalled.
Streaming checks use only the rules in `config.yaml`. Guardrails set up in the dashboard run only before the stream opens and after the response text has been sent, as described in [Streaming Guardrails](#streaming-guardrails), even when `guardrails.streaming.enabled` is `true`.
</Note>

---
Expand Down
6 changes: 3 additions & 3 deletions src/pages/docs/command-center/features/streaming.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ Each event contains a delta with the next token or function call. The stream end

## Streaming with Guardrails

Post-processing guardrails accumulate chunks as they stream. If a guardrail triggers in Enforce mode, the stream terminates immediately with an error. In Monitor mode, a warning is logged but streaming continues.
Post-processing guardrails run after the response text has been sent, just before the closing `[DONE]`, so they cannot stop or change the stream, and a slow check delays `[DONE]`. A block or warning is recorded in the request log. A self-hosted gateway can also check the stream while it is generated and stop it. See [Streaming Guardrails](/docs/command-center/features/guardrails#streaming-guardrails).

<Note>Pre-processing guardrails run before streaming begins. If they trigger in Enforce mode, the stream never starts.</Note>

Expand Down Expand Up @@ -222,8 +222,8 @@ Your application receives identical SSE events regardless of the underlying prov
<Card title="Caching" icon="database" href="/docs/command-center/features/caching">
Cache responses for faster repeated queries
</Card>
<Card title="Guardrails" icon="shield" href="/docs/command-center/features/guardrails">
Enforce policies during streaming
<Card title="Guardrails" icon="shield" href="/docs/command-center/features/guardrails#streaming-guardrails">
How guardrails treat streamed responses
</Card>
<Card title="Routing" icon="route" href="/docs/command-center/features/routing">
Route streaming requests across providers
Expand Down
6 changes: 3 additions & 3 deletions src/pages/docs/protect/concepts/understanding-protect.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,12 @@ description: "Protect screens your traffic for safety issues through guardrails
A **guardrail** wraps a check and adds three settings you configure in [Agent Command Center](/docs/command-center/features/guardrails), under **Gateway** > **Guardrails**:

- An **action** for when it triggers: Block, Warn, Mask, or Log
- A **stage** for when it runs: **pre** runs before the model sees the request, **post** runs before the response reaches the caller, and **both** runs on each side
- A **stage** for when it runs: **pre** runs before the model sees the request, **post** runs before the response reaches the caller (on a streamed response, after it has been sent; see [Streaming Guardrails](/docs/command-center/features/guardrails#streaming-guardrails)), and **both** runs on each side
- A **confidence threshold** for how sure the check has to be before it fires

A stage of **both** isn't a third mode. It's the same check running twice, once on the way in and once on the way out. Stage belongs to the guardrail rather than to the check it wraps, which is why a single **gateway**, the Agent Command Center surface your guardrails live on, can carry a mix: some watching only what the caller sent, others only what the model sent back.
A stage of **both** isn't a third mode. It asks for the same check to run twice, once on the way in and once on the way out. As a setting, stage belongs to the guardrail rather than to the check it wraps, which is why a single **gateway**, the Agent Command Center surface your guardrails live on, can carry a mix: some watching only what the caller sent, others only what the model sent back. Today the gateway doesn't apply the setting: it runs each check at its built-in stage, so Data Leakage Prevention and Hallucination Detection check the response, and the others check the request.

The threshold runs from 0.0 to 1.0: raising it makes the check fire only on higher-confidence matches, lowering it makes it fire more readily. Threshold and action are independent settings. The threshold decides how often a check fires; the action decides what the caller experiences when it does. Loosening a threshold on a check set to Log changes nothing a caller can see. Only Block changes the gateway's response status, to `403`; Warn and Log let the call return `200`. For the status each action produces, see [Guardrail checks](/docs/protect/reference/guardrail-checks#response-statuses).
The threshold runs from 0.0 to 1.0: raising it makes the check fire only on higher-confidence matches, lowering it makes it fire more readily. Threshold and action are independent settings. The threshold decides how often a check fires; the action decides what the caller experiences when it does. Loosening a threshold on a check set to Log changes only whether the call carries the `x-agentcc-guardrail-triggered` header. Today only a Block from a pre-stage check changes the gateway's response status, to `403`; a Block from a post-stage check, Warn, and Log let the call return `200`. For the status each action produces, see [Guardrail checks](/docs/protect/reference/guardrail-checks#response-statuses).

Every guardrail wraps a check, and the Rules tab groups checks under two headers that mark a single split: where the check runs. **Rule-Based Checks** are [first-party](/docs/protect/reference/guardrail-checks): they run inside Protect without an external provider. **AI-Powered Checks** call a provider you configure under **Provider Settings** in that check's own settings.

Expand Down
5 changes: 3 additions & 2 deletions src/pages/docs/protect/reference/guardrail-checks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,12 @@ These apply to every check on the gateway rather than to an individual check.

## Response statuses

These are the statuses the [Agent Command Center](/docs/command-center) gateway returns on a guardrail decision. Only the Block action changes the status; Warn and Log let the call through.
These are the statuses the [Agent Command Center](/docs/command-center) gateway returns on a guardrail decision. Today only a Block from a pre-stage check changes the status; a Block from a post-stage check, Warn, and Log let the call through. On a streamed response the status and headers are sent when the stream starts, so only a pre-stage check can change them; see [Streaming Guardrails](/docs/command-center/features/guardrails#streaming-guardrails).

| Action | Status | What the caller gets |
|---|---|---|
| Block | `403` | An error body with `"type": "guardrail_error"` and `"code": "content_blocked"`. A pre-stage check returns it before the model is ever called |
| Block, pre-stage check | `403` | An error body with `"type": "guardrail_error"` and `"code": "content_blocked"`, returned before the model is ever called |
| Block, post-stage check | `200` | The normal response, plus the header `x-agentcc-guardrail-triggered: true`. Only `data-leakage-prevention` and `hallucination-detection` run post-stage, whatever Stage is set to |
| Warn | `200` | The normal response, plus the header `x-agentcc-guardrail-triggered: true` |
| Log | `200` | The normal response, plus the header `x-agentcc-guardrail-triggered: true` |

Expand Down
Loading