Challenges and Risks
- Explain how hallucination compounds in multi-step agent tasks
- Describe three main challenges of running agents in production
- Identify the most common causes of cost escalation in agent deployments
The Gap Between Demo and Production
Agents perform well in controlled demos. They struggle in production, where inputs are messier, edge cases are real, and the cost of a mistake is actual. Understanding what goes wrong is the fastest way to design systems that go wrong less often.
Hallucination Compounds
Language models sometimes generate plausible-sounding but incorrect content — a problem known as hallucination. In a single-turn chatbot, one hallucination affects one answer. In an agent loop, one hallucination in an intermediate step can propagate through every subsequent step, compounding the error.
This is the central reliability problem for agentic systems. Researchers have noted that hallucination remains a persistent technical limitation for large models in high-stakes areas like healthcare and legal work. The solution is not to eliminate hallucination (which isn't possible with current models) but to verify claims through tool calls rather than relying on the model's internal knowledge.
Error Propagation in Multi-Step Tasks
When an agent runs 20 steps to complete a task, an error at step 5 affects every step that follows. The agent reasons from a flawed premise, calls tools with wrong arguments, and builds toward the wrong outcome — often confidently. By the time the final output is wrong, the trace may show 15 steps of coherent-but-mistaken reasoning built on one bad foundation.
Mitigations: validate tool outputs before using them in subsequent reasoning; build in explicit "sanity check" steps for critical intermediate results; and log enough context to reconstruct exactly where the error was introduced.
Reliability on Non-Trivial Tasks
A reliable finding from practitioners is that agents fail on non-trivial tasks in unpredictable ways. A task that works 95% of the time in testing fails 20% of the time in production, because real users phrase things differently and real data has edge cases that testing didn't cover. This isn't a solvable problem — it's a property of non-deterministic systems. The design response is to build in human escalation rather than assuming the agent will handle every case.
Debugging Difficulty
When something goes wrong with a traditional script, you read the error trace and find the line that failed. When something goes wrong with an agent, you review a sequence of 30 LLM calls to understand the reasoning path, identify where the model made the wrong decision, and determine whether the fix is in the prompt, the tools, the memory, or the model itself. This is fundamentally harder.
Good tracing (covered in the previous lesson) is the main tool for making this tractable. Without it, debugging a failed agent run is close to guesswork.
Cost Scales With Complexity
Every step in the agent loop consumes tokens. A 20-step task with tool calls and a large context window can cost 10–50x more than a single-turn response to the same question. In a multi-agent system, costs multiply further. For production deployments, understanding and managing cost per run is not optional — uncontrolled token usage is one of the most common reasons agent projects get shut down before they reach users.
Integration Complexity
Most real-world agent deployments involve connecting to existing systems: CRMs, databases, APIs, file systems. Each integration requires authentication, error handling, and format normalization. What looks like a simple tool call in a demo ("search the customer database") involves dozens of edge cases in production: the customer doesn't exist, the API rate limits, the schema changed, the response format varies by customer type. Underestimating integration work is a leading cause of project delays.
- Hallucination compounds in agents: one error in an intermediate step propagates through all subsequent reasoning — verify via tool calls, don't trust model memory
- Non-trivial agent tasks fail in production in ways that testing doesn't predict — human escalation is the design response, not better prompting alone
- Debugging an agent failure means reviewing 30+ LLM calls; without tracing, it's close to guesswork
- Cost scales with every loop iteration, tool call, and agent added — uncontrolled token usage shuts down more projects than reliability problems do
Progress is saved in this browser only — clear your cache or switch device and it's gone. Create a free account and it follows you everywhere.