Connecting an LLM: OpenAI, Claude, Gemini, and Ollama

The Chat Model Sub-Node

Every AI Agent node needs a Chat Model sub-node — the language model that powers the agent's reasoning. You connect it by clicking the Chat Model input on the agent node and selecting a model type. n8n supports over a dozen LLM providers, so you are not locked into any single vendor.

Major LLM Options in n8n

• OpenAI — GPT-4o, GPT-4o mini, o1, o3-mini. The most widely tested with n8n. GPT-4o is a strong default for production agentic workflows. API key from platform.openai.com.
• Anthropic (Claude) — Claude Opus, Sonnet, and Haiku. Excellent instruction-following and up to 200k token context windows. API key from console.anthropic.com.
• Google (Gemini) — Gemini 2.0 Flash and Pro. Fast, multimodal, and competitive on price. API key via Google AI Studio.
• Ollama — Run open-source models locally (Llama 3, Mistral, Qwen, Phi, and hundreds more). No API fees, full data privacy. Requires Ollama running on a server you control.
• Others — Groq (ultra-low latency), Azure OpenAI (enterprise), AWS Bedrock (multi-model), Mistral AI, Cohere.

Choosing a Model: Key Trade-offs

There is no universally best choice — the right model depends on your workflow's requirements:

• Capability — complex reasoning, multi-step tool use, and nuanced writing favour frontier models like GPT-4o or Claude Sonnet. Simple classification or extraction works well with smaller, cheaper models.
• Cost — frontier models charge per token. A classification agent running GPT-4o mini costs a fraction of one running GPT-4o. For high-volume automations, model selection is the largest cost lever.
• Latency — lighter models (GPT-4o mini, Gemini Flash, Groq-hosted models) return responses faster. For real-time chatbots, speed matters more than for batch workflows.
• Privacy — data sent to cloud LLM APIs is processed by that provider. For sensitive documents, Ollama (local inference) or an enterprise API agreement may be required.

Key Model Node Settings

After connecting a Chat Model sub-node, a few settings are worth understanding:

• Temperature — controls randomness. A value of 0 gives deterministic, consistent output — best for extraction and classification. Values of 0.7 to 1.0 produce more varied, creative responses. Most agentic workflows use 0 to 0.3.
• Max Tokens — caps output length. Leave at the provider default unless you need to constrain response size.
• Context Window — the maximum total tokens the model can process. Relevant when passing large documents or long conversation histories to the agent.

Credentials Setup

Each LLM provider needs its own credential in n8n. Go to Settings → Credentials → Add Credential, search for the provider, and paste your API key. Credentials are encrypted and reusable across all workflows in your instance — add them once and reference them from any model sub-node.

Using Multiple Models in One Workflow

You are not limited to one model per workflow. A common pattern is using a fast, cheap model (GPT-4o mini or Gemini Flash) for routing and simple tasks, then a capable frontier model only for the steps that genuinely need it. This keeps costs manageable without sacrificing quality where it counts.