Safety and Guardrails
- Explain the minimal footprint principle and why it matters
- List the four situations where human-in-the-loop checkpoints are most important
- Describe three types of scope limits that restrict agent behavior in production
Guardrails Are Not Optional
An agent with broad permissions and no oversight is not a powerful tool — it's a liability. The question isn't whether to add guardrails; it's how to design them so they don't eliminate the agent's usefulness while still bounding what it can do.
Well-designed guardrails are what separate a usable production agent from a demo that can't be safely deployed.
The Minimal Footprint Principle
The most important safety principle for agents: request only the permissions you need; prefer reversible actions over irreversible ones; and avoid storing sensitive information beyond what the current task requires.
This principle appears in Anthropic's guidance on building effective agents, in OWASP's LLM Top 10, and in most serious practitioner guides. It is the starting point for safe agent design.
Human-in-the-Loop Checkpoints
Human checkpoints are specific points in the agent workflow where execution pauses and a human must approve before the agent continues. They are most important before:
- Irreversible actions (deleting data, sending emails, making purchases, publishing to production)
- High-value transactions (above a defined dollar threshold)
- Actions affecting multiple users (broadcast messages, bulk updates)
- Novel situations the agent hasn't been tested on
Under the EU AI Act, human-in-the-loop requirements are mandatory for high-risk AI systems. The Act defines specific categories (employment, credit, legal proceedings, education access) where automated decisions require meaningful human oversight.
Scope Limits
Define exactly what the agent is allowed to do — and enforce it in code, not just in the prompt. Scope limits include:
- Tool restrictions — only provide the tools the agent needs for its specific task; remove tools when the task doesn't require them
- Data access controls — the agent reads from production databases, but only specific tables; it cannot modify schema
- API rate limits — cap the number of external API calls per run to prevent runaway behavior
- Query type restrictions — an analytics agent can run SELECT queries but not UPDATE, DELETE, or DROP
Sandboxing
Agents that run code should run it in a sandbox — an isolated execution environment that cannot affect the host system. Code generated by an LLM should never be executed with unrestricted file system or network access. Standard approaches include Docker containers with restricted capabilities or cloud-managed code execution environments.
Audit Logs
Every action an agent takes should be logged with the reasoning behind it: what the agent was trying to accomplish, which tool it called, what arguments it used, and what the result was. Audit logs serve three functions: post-incident investigation (what did the agent do and why?), compliance (demonstrating appropriate oversight), and improvement (finding patterns in what goes wrong).
OWASP LLM Top 10: Excessive Agency
OWASP's Top 10 for LLM Applications includes Excessive Agency as a critical risk: granting an agent more capabilities, permissions, or autonomy than necessary for its task. The recommended mitigation is the minimal footprint principle applied to tools, permissions, and data access — only grant what's needed, and verify that each permission is actually required before granting it.
- The minimal footprint principle: only the permissions needed, prefer reversible actions, don't store sensitive data beyond the task — this is the foundation of safe agent design
- Human checkpoints are mandatory before: irreversible actions, high-value transactions, actions affecting multiple users, and novel untested situations
- Scope limits must be enforced in code, not prompts — a tool restriction stated only in the system prompt can be argued around; a tool not provided cannot be called
- OWASP LLM Top 10 names Excessive Agency as a critical risk — the fix is minimal permissions plus verified necessity for each capability granted
Progress is saved in this browser only — clear your cache or switch device and it's gone. Create a free account and it follows you everywhere.