Connecting Local AI to Your Workflow

Local AI as a Drop-In Replacement

The most powerful aspect of Ollama’s and LM Studio’s OpenAI-compatible APIs is that they require almost no migration effort. Any tool, script, or application built for OpenAI can be redirected to your local model by changing one value: the base URL.

This applies to the OpenAI Python SDK, the OpenAI Node.js SDK, LangChain, LlamaIndex, Continue.dev, Cursor, and hundreds of other tools that support a custom API endpoint. The application does not need to know or care that it is talking to a local model instead of a cloud server.

Using Local Models in n8n

n8n has native AI nodes that connect to any OpenAI-compatible API. To use a local Ollama model in n8n:

Add an AI Agent node or an OpenAI Chat Model node to your workflow. In the credentials settings, create a new OpenAI credential and set the base URL to http://host.docker.internal:11434/v1 (if n8n is running in Docker) or http://localhost:11434/v1 (if running natively). Set the API key to any non-empty string. Set the model name to the Ollama model you want to use: llama3.3, qwen3, etc.

The workflow now calls your local Ollama model instead of OpenAI — same node, same workflow structure, zero API costs per execution. This is particularly valuable for high-volume automations where cloud API costs would accumulate.

Note: host.docker.internal is the hostname Docker containers use to reach services running on the host machine. If Ollama is also running in Docker on the same network, use the container name instead.

Sensitive Document Analysis

One of the clearest use cases for local AI is analysing documents that cannot be sent to a cloud service. The pattern is straightforward:

import ollama with open('contract.pdf', 'rb') as f: # Use a PDF library like pypdf to extract text first pass # extracted_text = your PDF extraction step response = ollama.chat( model='llama3.3', messages=[ {'role': 'system', 'content': 'You are a legal document reviewer. Identify unusual clauses and flag anything that deviates from standard commercial terms.'}, {'role': 'user', 'content': f'Review this contract: {extracted_text}'} ] ) print(response['message']['content'])

The document never leaves your machine. The model runs locally. No third-party has access to the content.

Running Ollama in Docker

For homelab or server deployments, Ollama has an official Docker image:

docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

The --gpus=all flag passes GPU access to the container (requires NVIDIA Container Toolkit on Linux). The -v ollama:/root/.ollama mounts a persistent volume so your downloaded models survive container restarts. After the container starts, use docker exec -it ollama ollama pull llama3.3 to pull models into the containerised Ollama.

On Apple Silicon, omit --gpus=all — Metal GPU access is not available inside Docker on macOS. For Apple Silicon homelab use, run Ollama natively (not in Docker) and let it use Metal directly.

The Development to Production Pattern

A practical pattern used by many teams in 2026: use Ollama locally for development, testing, and low-volume batch jobs, then route high-volume or user-facing production requests to a cloud API when local resources are saturated. Implement this with a simple environment variable:

import os from openai import OpenAI if os.getenv('USE_LOCAL') == 'true': client = OpenAI(base_url='http://localhost:11434/v1', api_key='ollama') model = 'llama3.3' else: client = OpenAI(api_key=os.getenv('OPENAI_API_KEY')) model = 'gpt-4o'

Switch between local and cloud with a single environment variable. The rest of your code is identical.