Learn › ComfyUI Advanced: Training, Kontext, and Automation › VRAM and Performance Optimization

VRAM and Performance Optimization

Advanced 🕐 12 min Lesson 11 of 12
What you'll learn
  • Explain the VRAM cost of each major model component in a FLUX workflow and when VRAM pressure causes generation to slow
  • Load a FLUX model in FP8 precision using the Load Diffusion Model node with weight_dtype set
  • Install the GGUF loader and choose the right quantization tier for a given GPU VRAM size
  • Apply the correct ComfyUI startup flag for each offloading scenario: --lowvram, --medvram, or --gpu-only

How ComfyUI Uses VRAM

ComfyUI loads models into VRAM when they are first needed and keeps them there until VRAM pressure forces them out. Understanding this cycle helps you predict when you will run out of memory and what to do about it.

The models competing for VRAM in a typical advanced workflow:

  • Diffusion model: FLUX.1 [dev] in BF16 is approximately 23 GB; in FP8 approximately 12 GB; in GGUF Q4_K_S approximately 7 GB
  • VAE: approximately 350 MB
  • Text encoders (CLIP L + T5 XXL): approximately 9 GB in BF16; approximately 5 GB in FP8
  • ControlNet: approximately 6 GB for FLUX ControlNet in BF16; approximately 3 GB in FP8
  • LoRA: negligible VRAM when loaded as a patch; some overhead when applied during generation

A workflow using FLUX.1 dev BF16 + text encoders BF16 + a ControlNet exceeds 38 GB VRAM. On a 24 GB GPU, ComfyUI will page models in and out of RAM, which causes generation to slow dramatically. On a 16 GB GPU, some workflows simply will not run without quantization.

FP8 Quantization

FP8 (8-bit floating point) reduces a BF16 model's VRAM footprint by roughly half, with minimal quality loss on most subjects. It is the fastest way to fit large models onto 16–24 GB GPUs without changing the workflow structure.

For FLUX, Black Forest Labs provides official FP8 model files:

  • flux1-dev-fp8.safetensors — approximately 12 GB; available on Hugging Face at black-forest-labs/FLUX.1-dev
  • flux1-dev-kontext_fp8_scaled.safetensors — the FLUX Kontext model in FP8 (this is the version used throughout this course)

To use an FP8 model in ComfyUI, load it with the Load Diffusion Model node exactly as you would a BF16 model. No node changes are required. Set the weight_dtype input on that node to fp8_e4m3fn (or fp8_e5m2 — the difference is minor; e4m3fn is the more common choice for FLUX).

The text encoders can also be loaded in FP8: download t5xxl_fp8_e4m3fn.safetensors (or the scaled variant used in this course: t5xxl_fp8_e4m3fn_scaled.safetensors) and use it in the DualCLIPLoader node. CLIP L is already small and does not benefit much from FP8 quantization; the T5 XXL reduction from roughly 8 GB to roughly 5 GB is significant on a 16 GB GPU.

Quality impact of FP8: for photorealistic and illustrative outputs, the difference from BF16 is not visible at standard viewing sizes. For very fine-grained texture work or multi-step iterative editing with Kontext, BF16 produces marginally sharper intermediate outputs. If quality matters more than VRAM, use BF16; if VRAM is the constraint, FP8 is the right choice.

GGUF Quantization

GGUF is a quantization format originally designed for CPU inference (llama.cpp) that has been adapted for image models. GGUF files compress the model weights more aggressively than FP8 — a Q4_K_S FLUX model is approximately 7 GB — but require a custom node to load.

Install the GGUF loader: in ComfyUI-Manager, search for ComfyUI-GGUF (by city96) and install it. After restarting ComfyUI, a new node appears: Unet Loader (GGUF).

Download GGUF quantizations of FLUX from Hugging Face: search for "FLUX.1-dev-gguf" — city96 maintains a widely-used set of quantized files. The Q4_K_S variant (approximately 7 GB) is the best quality-per-VRAM option for 12–16 GB GPUs. Q8_0 (approximately 14 GB) is very close to FP8 quality. Q2_K (approximately 4 GB) fits on 8 GB GPUs but shows noticeable quality degradation on detailed subjects.

Workflow change: replace the standard Load Diffusion Model node with Unet Loader (GGUF) and select your GGUF file. All other nodes remain the same. The output is wired identically.

Quality vs VRAM summary:

  • BF16: full quality, 23 GB — requires 24 GB+ GPU
  • FP8: near-full quality, 12 GB — fits comfortably on 16 GB GPU
  • GGUF Q8_0: near-FP8 quality, 14 GB — fits on 16 GB GPU with less headroom
  • GGUF Q4_K_S: good quality, 7 GB — fits comfortably on 12 GB GPU
  • GGUF Q2_K: reduced quality, 4 GB — 8 GB GPU viable, detail loss visible

Model Offloading Flags

ComfyUI provides command-line flags that control how aggressively it moves models between VRAM and system RAM. These flags trade speed for VRAM headroom.

Start ComfyUI with these flags to change offloading behaviour:

  • --lowvram: moves the entire model to system RAM between generation steps, loading only the layer currently needed into VRAM. This allows FLUX to run on a 6–8 GB GPU, but generation is very slow (5–10× slower than on a 24 GB GPU at full speed) because of constant VRAM-to-RAM transfers.
  • --medvram: loads the full model into VRAM for inference but offloads it to RAM immediately after the generation step finishes. If you are running multiple tools in sequence (ControlNet → FLUX → upscaler), each loads into VRAM when needed and offloads when done. Use this if your GPU is 12–16 GB and you are running workflows with multiple large models.
  • --gpu-only: opposite of offloading — forces everything to stay in VRAM at all times, never moving to RAM. Only use this if you have enough VRAM to hold all models simultaneously and want to eliminate the latency of model loading between generations.
  • --disable-smart-memory: turns off ComfyUI's automatic model eviction algorithm. Use this only if you are seeing unexpected model reloads in a workflow that you know fits in VRAM — it prevents ComfyUI from evicting a model that is still needed.

Add these flags when starting ComfyUI: python main.py --lowvram or python main.py --medvram.

VAE Tiling

The VAE (variational autoencoder) decodes the latent image into pixels at the end of generation. For images above roughly 2048px, the VAE decode step itself can run out of VRAM even if the generation step completed fine.

The VAE Decode (Tiled) node solves this by decoding in overlapping tiles. In ComfyUI-Manager, search for ComfyUI-TiledVAE or look for the built-in VAE Decode (Tiled) node. Replace the standard VAE Decode node with the tiled version and set:

  • tile_size: 512 is safe on 8–12 GB GPUs; 1024 is safe on 16–24 GB GPUs; larger tiles decode faster
  • overlap: 64–128 pixels prevents visible seams at tile boundaries — always set this

VAE Decode (Tiled) adds a few seconds to generation time but eliminates out-of-memory errors at high resolution. Use it for any workflow targeting output above 2048×2048 pixels.

Batch Size and Its Effects

ComfyUI's Empty Latent Image node has a batch_size input. Setting it above 1 generates multiple images in parallel in a single forward pass. VRAM usage scales linearly with batch size — running batch 4 on a 24 GB GPU at BF16 will not work unless the base generation at batch 1 leaves at least 75% of VRAM free.

Batch size is useful when:

  • Generating variations at the same prompt (batch 4 is faster than 4 sequential runs)
  • Running parameter sweeps where you need to compare multiple outputs side by side

Batch size is not useful when:

  • VRAM is already constrained at batch 1
  • Each generation needs a unique seed or prompt — in this case, sequential runs with different seeds are cleaner than batching

At batch 1, VRAM usage is predictable and stable. At higher batch sizes, VRAM usage can spike unpredictably during the attention computation. If you want to use batch sizes above 1, stay at the FP8 precision tier or use GGUF quantization to leave headroom.

Torch Compile and Attention Optimisation

ComfyUI automatically selects the fastest attention implementation available on your GPU (Flash Attention 2, xFormers, or PyTorch's built-in SDPA). No manual configuration is needed in most cases.

If you have xFormers installed (pip install xformers), ComfyUI will use it automatically and you will see a small speed improvement and reduced VRAM usage on SDXL workflows. For FLUX, xFormers benefit is smaller than on SDXL, but it does not hurt.

Torch compile (torch.compile()) can speed up generation by 20–40% after a one-time compilation overhead. Enable it in ComfyUI by starting with --fast (a ComfyUI flag that enables a range of speed optimisations including torch compile on supported GPUs). The first generation after starting takes longer as the model compiles; subsequent runs are faster. Supported on CUDA GPUs with PyTorch 2.0+.

Diagnosing Slow Generations

If generation is slower than expected, the likely causes and fixes:

  • Models paging to RAM: ComfyUI prints a message when a model is moved to RAM. If you see these messages, your workflow exceeds your VRAM. Solution: switch to FP8 or GGUF for the largest models.
  • Slow first run, fast subsequent runs: normal — models are loaded from disk on first use and cached in VRAM or RAM afterward. This is not a bug.
  • Python overhead between nodes: ComfyUI processes each node sequentially with a small overhead per node. For very large graphs (100+ nodes), this overhead adds up. Use the ComfyUI efficiency nodes pack (via Manager) to merge common node groups and reduce graph size.
  • High step count: FLUX is designed to work well at 20–28 steps with Euler or Euler Ancestral sampler. SDXL works at 20–30 steps. Increasing steps beyond these ranges produces diminishing returns at linear time cost.
Key takeaways
  • FLUX.1 dev in BF16 is 23 GB; in FP8 it drops to 12 GB; in GGUF Q4_K_S to 7 GB — the right precision tier is determined by GPU VRAM, not preference
  • FP8 is the first choice for VRAM reduction: load the official BFL FP8 model files with weight_dtype fp8_e4m3fn on the Load Diffusion Model node — no other workflow changes needed
  • The GGUF loader (city96/ComfyUI-GGUF) enables Q4_K_S at 7 GB for 12 GB GPUs — replace Load Diffusion Model with Unet Loader (GGUF) and wire identically
  • VAE Decode (Tiled) with tile_size 512 and overlap 64 eliminates OOM errors during the decode step for outputs above 2048px
  • --lowvram allows FLUX to run on 6-8 GB GPUs but is 5-10x slower than native VRAM execution due to constant VRAM/RAM transfers
Learn more every week Get 5 hand-picked prompts + new AI tools every Monday. Free.