Use your own guard model

A guard model is a safety model you run, such as NVIDIA Nemotron Safety Guard. With it on, Forgebench sends each prompt, each response, or both to your model before letting the call through. Your model decides what is unsafe. You decide what happens for each kind of unsafe content.

The guard model is one of the checks under Guardrails > Inline enforcement. Unlike guardrail findings, these checks run during the call and can block it.

Requires the Pro plan or higher. Changing the setup needs Admin, or a role with the Edit guardrails permission.

What you need

• A Nemotron guard model behind an OpenAI-compatible API. Forgebench calls POST <base URL>/chat/completions. NVIDIA NIM serves this API, both hosted by NVIDIA and in your own cluster. These models are supported:

• An address Forgebench can reach. On Forgebench cloud, that means a public HTTPS URL. Private addresses and plain http are refused unless the host is on the deployment's allowlist (OPENAI_COMPATIBLE_ALLOWED_HOSTS on the control plane). Loopback and cloud metadata addresses are always refused.

Set it up

1. Serve the model

   Run the guard model on NVIDIA NIM, or use NVIDIA's hosted API at https://integrate.api.nvidia.com/v1. Note the base URL, which usually ends in /v1, and the model id, for example nvidia/llama-3.1-nemotron-safety-guard-8b-v3.

2. Connect it as an endpoint

   This needs the admin role. In Admin > API keys > Credentials, click Add credential, then OpenAI-compatible endpoint…. Enter a Name (for example nim), the Base URL, an API key if your server needs one, and the model id under Models. The model is then called oc/nim/<model id>. Admins can also do this from the + button next to the model picker in step 4.

3. Open Inline enforcement

   Go to Guardrails and open the Inline enforcement tab.

4. Turn on Guard model

   Set Guard model to Log or Block. Then pick the Model and its Type, and choose whether to check Prompts, Responses or both.

5. Test it, then save

   Try a few prompts in Test, then click Save. If the change can block calls, you are asked to confirm first.

Settings

Category rules

A rule matches the category exactly as your model reports it:

• Nemotron Safety Guard v3 reports names, such as Violence or Unauthorized Advice.
• Nemotron 3.5 content safety often reports no category, so the mode applies.

When several categories come back, the strictest result wins: Block, then Log, then Allow. A category without a rule uses the mode. An unsafe verdict with no category also uses the mode.

For example, a health app might allow Unauthorized Advice, block Violence, and log everything else.

Strict mode

The model may be down, time out, or reply in a way Forgebench can't read. In that case:

• Strict on: the call is blocked.
• Strict off: the call goes through unchecked.

With Strict on, an outage of your guard model blocks every call. Watch its uptime.

With Strict off, the audit log gets a guardrail_health.model_guard.unavailable entry while the model can't answer, at most one every 5 minutes, so an outage doesn't go unnoticed.

Test before you save

Test runs your model on a prompt, and optionally a response, with the settings on screen, saved or not. It shows Safe, Allowed, Logged or Blocked, the categories, how long the model took, and its raw reply. Nothing is recorded and nothing is blocked.

What a blocked caller gets

The call fails with HTTP 400:

{
  "detail": {
    "code": "guardrail_blocked",
    "guardrail": "model_guard",
    "direction": "input",
    "message": "blocked by a content-safety guardrail",
    "reason": "guard model flagged input as unsafe: Violence"
  }
}

direction is input for a prompt and output for a response. Every blocked or logged call is also recorded in the audit log.

Limits

• When responses are checked and can be blocked, including with Strict on, streaming requests (stream: true) are refused. To keep streaming, check prompts only, or use Log.
• A call blocked on its response is still billed, because the model already ran.
• Calls to your guard model are not metered and don't count toward budgets.
• The guard model reads every user and tool message, not just the latest. A message it has already judged is remembered for a day (as a hash, not the text) and isn't sent again, so each turn sends only its new messages. Changing the guard's settings judges everything again.
• New text over 32,000 characters is checked in parts, each one its own guard call. Text needing more than 8 parts counts as an error, so Strict blocks it.
• Chat completions and calls from coding tools such as Claude Code, Cursor and Codex are checked. Embeddings, batches and voice calls are not, and neither are the calls Forgebench makes for its own features (routing, knowledge-base answers, insights, evals).

Keep prompts inside your network

On Forgebench cloud, prompts pass through Forgebench before reaching your guard model. If that isn't allowed, run Forgebench in your own cluster (BYOC or air-gapped, see Billing). Your agents, Forgebench and your guard model then all stay inside your network, and the guard model can sit on a private address on the allowlist.

Next