Learn › AI Agents: Building & Production › Safety and Guardrails

Safety and Guardrails

Intermediate 🕐 12 min Lesson 6 of 7
What you'll learn
  • Explain the minimal footprint principle and why it matters
  • List the four situations where human-in-the-loop checkpoints are most important
  • Describe three types of scope limits that restrict agent behavior in production

Guardrails Are Not Optional

An agent with broad permissions and no oversight is not a powerful tool — it's a liability. The question isn't whether to add guardrails; it's how to design them so they don't eliminate the agent's usefulness while still bounding what it can do.

Well-designed guardrails are what separate a usable production agent from a demo that can't be safely deployed.

The Minimal Footprint Principle

The most important safety principle for agents: request only the permissions you need; prefer reversible actions over irreversible ones; and avoid storing sensitive information beyond what the current task requires.

This principle appears in Anthropic's guidance on building effective agents, in OWASP's LLM Top 10, and in most serious practitioner guides. It is the starting point for safe agent design.

Human-in-the-Loop Checkpoints

Human checkpoints are specific points in the agent workflow where execution pauses and a human must approve before the agent continues. They are most important before:

  • Irreversible actions (deleting data, sending emails, making purchases, publishing to production)
  • High-value transactions (above a defined dollar threshold)
  • Actions affecting multiple users (broadcast messages, bulk updates)
  • Novel situations the agent hasn't been tested on

Under the EU AI Act, human-in-the-loop requirements are mandatory for high-risk AI systems. The Act defines specific categories (employment, credit, legal proceedings, education access) where automated decisions require meaningful human oversight.

Scope Limits

Define exactly what the agent is allowed to do — and enforce it in code, not just in the prompt. Scope limits include:

  • Tool restrictions — only provide the tools the agent needs for its specific task; remove tools when the task doesn't require them
  • Data access controls — the agent reads from production databases, but only specific tables; it cannot modify schema
  • API rate limits — cap the number of external API calls per run to prevent runaway behavior
  • Query type restrictions — an analytics agent can run SELECT queries but not UPDATE, DELETE, or DROP

Sandboxing

Agents that run code should run it in a sandbox — an isolated execution environment that cannot affect the host system. Code generated by an LLM should never be executed with unrestricted file system or network access. Standard approaches include Docker containers with restricted capabilities or cloud-managed code execution environments.

Audit Logs

Every action an agent takes should be logged with the reasoning behind it: what the agent was trying to accomplish, which tool it called, what arguments it used, and what the result was. Audit logs serve three functions: post-incident investigation (what did the agent do and why?), compliance (demonstrating appropriate oversight), and improvement (finding patterns in what goes wrong).

OWASP LLM Top 10: Excessive Agency

OWASP's Top 10 for LLM Applications includes Excessive Agency as a critical risk: granting an agent more capabilities, permissions, or autonomy than necessary for its task. The recommended mitigation is the minimal footprint principle applied to tools, permissions, and data access — only grant what's needed, and verify that each permission is actually required before granting it.

Key takeaways
  • The minimal footprint principle: only the permissions needed, prefer reversible actions, don't store sensitive data beyond the task — this is the foundation of safe agent design
  • Human checkpoints are mandatory before: irreversible actions, high-value transactions, actions affecting multiple users, and novel untested situations
  • Scope limits must be enforced in code, not prompts — a tool restriction stated only in the system prompt can be argued around; a tool not provided cannot be called
  • OWASP LLM Top 10 names Excessive Agency as a critical risk — the fix is minimal permissions plus verified necessity for each capability granted
Learn more every week Get 5 hand-picked prompts + new AI tools every Monday. Free.