Capstone: Build Your Personal Local AI Setup

Your Personal Setup Decision Tree

This capstone is a practical guide. Work through it from top to bottom and you will leave with a working local AI setup matched to your hardware.

Step 1: Identify Your Hardware Tier

Run this check before choosing any model or tool:

• 8 GB RAM, no dedicated GPU: CPU-only inference. Expect 2–5 tokens/sec. Best models: Phi-4-mini (3.8B), Gemma 3 1B. Use LM Studio for UI; Ollama for scripting.
• 8 GB VRAM: Entry-point GPU experience. 25–40 tokens/sec. Best models: Phi-4-mini, Llama 3.3 8B Q4, Qwen3 7B Q4. All tools work well.
• 12 GB VRAM: The sweet spot. Full Q8 quality on 7–8B models; Q4 on 14B. Best all-round experience. Any of the main model families fits.
• 16 GB+ VRAM: 24B models at Q4; 32B with partial CPU offloading. For power users: Qwen3 32B, Llama 3.3 70B with CPU offload.
• Apple Silicon (any M-series): All RAM acts as VRAM. 16 GB Apple = comfortable 13B models; 32 GB = 30B+ models. Use MLX format in LM Studio for fastest inference.

Step 2: Choose Your Stack

Just want to chat: LM Studio only. Download, install, browse models, chat.

Developer / automation: Ollama + Open WebUI. Ollama serves the API; Open WebUI gives you a browser UI with history.

Power user: All three — LM Studio for model discovery, Ollama for scripting/production, Open WebUI for daily chat. They coexist on separate ports with separate model storage.

Step 3: Pull Your Starting Models

For a good baseline that covers most tasks on 8–12 GB VRAM:

ollama pull llama3.3 # general chat, summarisation ollama pull qwen3 # coding tasks ollama pull phi4-mini # fast, lightweight tasks ollama pull llava # image analysis

Step 4: Create One Modelfile

Pick the task you will use local AI for most, write a Modelfile for it, and register it with ollama create. Having a pre-configured assistant for your primary use case removes setup friction from every session.

Troubleshooting: The Most Common Problems

Problem: Inference is very slow (2–5 tokens/sec on a GPU machine)

Cause: the model is running on CPU instead of GPU. Check with ollama ps — if your GPU name does not appear, GPU is not being used.

Fix on macOS: Uninstall and reinstall Ollama using the .app installer from ollama.com (not Homebrew). Rosetta 2 binaries skip Metal.

Fix on AMD Linux: Set HSA_OVERRIDE_GFX_VERSION=10.3.0 (RDNA 2) or 11.0.0 (RDNA 3) before starting Ollama.

Fix on NVIDIA: Update to the latest NVIDIA driver and restart Ollama.

Problem: The model gives short, confused, or truncated responses

Cause: almost always model size vs hardware mismatch. When a model spills from VRAM into system RAM, not only does it slow down — context management degrades. The model loses track of earlier parts of the conversation.

Fix: use a smaller model that fits entirely in VRAM, or reduce the context window with PARAMETER num_ctx 2048 in a Modelfile. A faster small model will feel smarter than a slow large model that cannot hold context.

Problem: Ollama API returns connection refused

Cause: the Ollama server is not running. On macOS it runs as a menu bar app — check the top right menu bar. On Linux, run ollama serve in a terminal or configure it as a systemd service. On Windows, check that the Ollama service is running in Task Manager.

Problem: LM Studio API returns 404 or connection refused

Cause: the server was not started, no model is loaded, or the port is wrong. Go to the Developer tab in LM Studio and confirm the server is running (green indicator). Ensure a model is loaded in the Chat tab. The correct port is 1234, not 11434.

Problem: Downloaded model file is corrupted or model crashes on load

Cause: interrupted download. In Ollama, remove the model and re-pull it: ollama rm llama3.3 && ollama pull llama3.3. In LM Studio, delete the model from the My Models tab and re-download it from Discover.

When Local AI Is Not the Right Choice

Local AI is not always better. Cloud models are the right choice when: you need the absolute best reasoning quality (Opus 5.5, GPT-6 Astra), you are handling tasks that need real-time web access, you are on mobile or a device without enough RAM, or you need a response in under one second for a user-facing product. The power of knowing both is knowing when to use each.