ElevenAgents: Build Conversational Voice Agents

From Audio Generation to Live Conversation

All the tools covered so far in this track produce pre-rendered audio: you provide text, the model generates a file. ElevenAgents does something fundamentally different — it powers live, real-time voice conversations between a human and an AI that listens, thinks, and speaks back.

ElevenAgents is ElevenLabs' platform for building and deploying conversational voice agents. A voice agent is an AI that can hold a spoken conversation: it hears what you say, understands the intent, generates a response, and delivers it in a natural-sounding voice — all in real time. The round-trip happens fast enough to feel like a real conversation.

The Visual Builder

ElevenAgents provides a visual builder that lets you define agent behaviour without writing complex code. The core components you configure are:

• Voice — Pick any voice from your library. This is the voice your agent will speak in for every response.
• System prompt — A natural language description of the agent's role, personality, scope, and any rules it should follow. This is the most important configuration — it determines how the agent behaves.
• Knowledge base — Documents, FAQs, or structured data the agent can reference to answer questions accurately.
• Tools and integrations — External APIs the agent can call during a conversation: look up a calendar, query a database, create a ticket, send a message.
• Guardrails — Rules about what topics the agent should decline to discuss or escalate to a human.

Deployment Channels

Once you have built an agent, you deploy it to a channel — the surface where users will interact with it. ElevenAgents supports multiple channels:

• Web widget — An embeddable widget you add to any webpage. Users click a microphone button and start speaking.
• Phone (via Twilio and similar) — Connect a phone number so your agent can receive or make phone calls.
• WhatsApp — Deploy the agent as a WhatsApp contact that responds to voice messages.

Different channels suit different use cases. A customer support agent for a SaaS product usually starts as a web widget. A restaurant reservation agent might live on a phone number that customers can call.

Emotionally Intelligent Responses

ElevenAgents incorporates emotionally aware response generation. The agent models not just what to say but how to say it — adjusting tone, pacing, and vocal delivery based on the emotional context of the conversation. A customer expressing frustration gets a calmer, more measured response. A user asking about exciting news gets a more energised delivery. This is one of the platform's differentiators compared to generic voice bots.

Common Agent Use Cases

Try This

In ElevenAgents, create a new agent with a simple brief: a customer support agent for a fictional coffee subscription service. Give it a warm, friendly voice from the library. Write a system prompt that defines its role (help customers manage their subscription), its personality (helpful, knowledgeable, not pushy), and one rule (always confirm the customer's email before making any account changes). Enable the web widget deployment. Start a test conversation and ask it a question your system prompt should cover — then ask something outside its scope and see how it handles the guardrail.