Vector Stores and Retrieval: Making Your Agent Smart

What Vector Embeddings Are

A vector embedding is a list of numbers — typically 1,500 to 3,000 values — that represents the meaning of a piece of text. Two chunks discussing the same concept will have similar vectors even if they use completely different words. This is what makes semantic search possible: instead of matching keywords, vector search finds documents that are conceptually close to the query, even when the exact terms do not appear.

How Semantic Retrieval Works

When a user asks a question, the RAG query workflow runs these steps:

1. Converts the question into a vector embedding using the same model used during ingestion.
2. Queries the vector database for the N chunks whose vectors are most similar to the question vector.
3. Returns those chunks as text — the most semantically relevant passages from your knowledge base.
4. Passes the retrieved text to the AI Agent as context alongside the user's question.

The agent then generates an answer grounded in the retrieved content rather than relying purely on its training data.

Vector Stores Supported in n8n

• Pinecone — fully managed, cloud-native vector database. Scales automatically, popular for production deployments. No infrastructure to manage.
• Qdrant — open-source, available as self-hosted or managed cloud. Strong metadata filtering capabilities alongside vector search.
• Supabase pgvector — adds vector search to a Postgres database you may already have in Supabase. Reduces the number of services to manage.
• Postgres pgvector — the pgvector extension on a self-hosted Postgres instance. Best if you already run Postgres and want to keep everything in one database.
• Weaviate — open-source, feature-rich, supports hybrid search (combining keyword and semantic matching in one query).
• Milvus — high-performance, purpose-built for vector workloads at large scale.

The Vector Store Tool Sub-Node

To connect your vector store to an AI Agent, use the Vector Store Tool sub-node. It wraps the semantic search query as a tool the agent can call. The agent invokes it automatically when it determines a knowledge base lookup is appropriate. Add a description that makes this clear:

"Search the product documentation and support knowledge base for relevant information. Use this when answering any question about features, pricing, troubleshooting steps, or product behaviour."

The Critical Rule: Consistent Embeddings

The embedding model used when storing vectors must exactly match the model used when querying. If you indexed your documents with text-embedding-3-small, every query must also use text-embedding-3-small. Vectors generated by different models live in incompatible dimensional spaces — using a different model at query time makes all similarity comparisons meaningless and returns effectively random results.

Retrieval Parameters to Tune

• Top K — how many chunks to retrieve per query. Default is 4. Too few and the agent may miss relevant context. Too many and the context window fills with noise.
• Similarity Threshold — minimum similarity score required to include a result. Raise this to get fewer, higher-quality results when precision matters more than recall.