What are AI agent guardrails?

Guardrails are the limits on what an AI agent is allowed to do: which tools it can call, with which arguments, and which actions need a human first. Guardrails that hold are enforced by the system around the agent, not just described in its prompt.

Can you put guardrails in the system prompt?

You can, and it is worth writing the intent there, but prompt instructions are advisory. The model can be talked out of them by adversarial input or lose track of them over a long run. Treat prompt rules as reminders and enforce the real limits server-side.

Where should AI agent guardrails be enforced?

On the server, between the agent and the action. A server-side policy that inspects each proposed tool call and returns allow, deny, or require-approval is the only layer that holds regardless of the prompt or the client. Prompt-level and client-level checks are bypassable.

What is the difference between guardrails and output validation?

Output validation checks what the model says, scanning responses for PII, toxicity, or invalid formats. Action guardrails check what the model is allowed to do, authorizing or refusing tool calls before they run. They solve different problems, and production agents need both.

How do you let an AI agent take real actions safely?

Give it a small set of narrowly scoped tools, enforce their limits with a server-side policy, require human approval for high-impact actions, and log every attempt, allowed or refused. On OBTO the policy layer and the Glass Receipt are built in, so an agent can act in production without holding unchecked power.

← All articles Security

Your AI agent's guardrails shouldn't live in the prompt

OBTO Team · Insights from the Glass Box

A guardrail you write into a prompt is a request. "Never delete a record without confirming." "Only refund up to $50." "Don't touch another tenant's data." The model reads those the same way it reads everything else — as text to weigh, not a rule to obey. Most of the time it complies. The problem is the times it doesn't, and that you can't tell in advance which times those will be.

An agent that takes real actions needs guardrails that hold when the model is wrong: jailbroken by a hostile input, drifting after forty steps, or simply confident about a bad plan. That means moving the guardrail out of the prompt and into a place the model can't argue with. Here is what that looks like.

The three places a guardrail can live

There are only three layers where you can limit what an agent does, and two of them leak.

In the prompt (advisory)

Instructions in the system prompt are the easiest to write and the easiest to bypass. They depend entirely on the model choosing to follow them. A cleverly worded ticket, a poisoned document sitting in the context, or a run long enough to push the instruction out of the model's attention — any of these can turn "never do X" into X. Prompt rules are worth having. They are not a control.

In the client (bypassable)

The application calling the model can do more: inspect a tool's arguments before executing it, block a call that looks wrong. Better — but the guarantee is only as strong as that one client. Point a second client at the same tools, or let the agent reach the tool directly, and the check simply isn't there. Anything holding the credentials can walk around a guardrail that exists in only one caller.

On the server, in front of the action (enforced)

The only guardrail that isn't optional is the one that runs on the server, ahead of the action itself. The agent proposes: call this tool, with these arguments. A policy on the server decides — allow, refuse, or require a human — before anything happens. The model's decision is an input to that policy. It is never the final word. This is the layer that still holds when everything above it fails.

This is a different job from the guardrail libraries that scan a model's output for PII, toxicity, or malformed JSON. Those are genuinely useful: they check what the model says. Action guardrails check what the model is allowed to do. You want both — just don't mistake one for the other.

What "enforced" actually takes

Four moves turn a good intention into a guardrail that survives a bad day.

1. Name the exact actions

The first guardrail is scope. Don't give an agent "database access"; give it refund_order(order_id, amount) with the amount capped server-side. Don't hand it raw SQL; expose the three queries it actually needs as typed tools. An agent can only misuse the actions you handed it, so hand it few, and make each one narrow. We walked through this for databases in connecting Postgres without giving it raw SQL and, more generally, in the guide to building an MCP tool. The tool is the boundary.

2. Enforce it server-side

Put the decision somewhere the model can't reach it. On OBTO that's a policy that runs on the server before a tool executes: it sees the proposed call and its arguments and returns allow, deny, or needs-approval. Because it runs server-side, it applies no matter which client made the call, no matter what the prompt said, and no matter how the model was talked into asking. The enforcement lives outside the model on purpose.

3. Gate the risky actions on approval

Not every action should be fully autonomous. Reading a ticket, drafting a reply, looking up an order — let those run. Issuing the refund, deleting the account, emailing every customer — those can wait three seconds for a person. A policy that flags actions above a threshold and holds them for approval lets you turn autonomy up on the safe majority without betting the dangerous few on the model having a good day. It's the same argument we made for helpdesk agents that resolve rather than just draft.

4. Receipt every attempt

Every action — allowed or refused — should leave a record. OBTO's Glass Receipt logs each tool call the agent made, each one the policy blocked, the arguments, and the outcome. A refused action is as visible as a completed one, which is exactly what you want when you're deciding whether to trust the agent with more. Guardrails you can't see are guardrails you can't tune; there is more on that in building an audit trail your agent can't skip. A single decision on the receipt reads about like this:

{
  "run_id": "run_c41d",
  "proposed": { "tool": "refund_order", "args": { "order_id": "A-8842", "amount": 240.00 } },
  "policy": "refunds_over_100_need_approval",
  "decision": "held_for_approval",
  "approver": "pending",
  "executed": false
}

Why agents raise the stakes

A prompt guardrail survives the demo. The demo has one cooperative user, clean input, and a short task. Production adds adversarial input, tenants who must never see each other's data, and runs long enough that the model forgets where it started. That's the same gap we keep returning to in why AI agents break in production, and it's why enforcement has to live somewhere the model's mood can't reach. A control that works only when the model behaves isn't a control; it's a hope.

It is also why "the agent holds the keys" is the wrong default — a close cousin of this problem we covered in why your agent should never hold its own keys. Scope, server-side policy, approval, and a receipt are the same idea applied to actions instead of secrets: the agent gets to ask, and something outside the agent decides.

The short version

Write the friendly reminders in the prompt if you like. Then assume the model will ignore them, and build the real guardrail underneath: name the exact actions, enforce the limits server-side, gate the dangerous ones on a human, and receipt every attempt. That is the line between an agent you demo and one you let touch production. To see it working on a real workflow, the getting-started guide takes about ten minutes, and OBTO's pricing includes the policy layer and the Glass Receipt on every tier. The guardrails aren't the paid add-on.