VAE, CLIP, and Conditioning: What They Actually Do

The Pipeline You've Been Running, Explained

Every time you press Ctrl+Enter, the same sequence of operations happens: your text prompt travels through CLIP and becomes conditioning, the KSampler uses that conditioning to guide a denoising process in latent space, and the VAE converts the result back to pixels. These three systems — VAE, CLIP, and conditioning — are the hidden machinery behind every image ComfyUI generates.

Most users spend weeks running workflows without understanding what these nodes do. That creates a real problem: when something looks wrong (muddy colors, prompt being ignored, quality degradation), you have no mental model for diagnosing it. This lesson fixes that.

Latent Space: Why It Exists

Running the diffusion process directly on full-size pixel images would be extremely slow and memory-intensive. Instead, Stable Diffusion (and FLUX.1) work in a compressed "latent space" — a lower-dimensional representation of the image that is much smaller than the pixel version but preserves the essential visual information.

The latent representation of a 512x512 pixel image is only 64x64 in four channels — about 64 times smaller than the pixel image. This is why the KSampler works with LATENT data rather than IMAGE data, and why image generation is much faster than working pixel-by-pixel would be.

The VAE is the bridge between the two spaces.

The VAE: Encoder and Decoder

VAE stands for Variational AutoEncoder. It has two functions in a generation workflow:

• Encode: Convert a pixel image into latent space. Used in image-to-image and inpainting workflows when you start from an existing image. The VAE Encode node performs this step.
• Decode: Convert a latent image back into pixels. Used in every text-to-image workflow. The VAE Decode node performs this step, turning the KSampler's LATENT output into an IMAGE you can see and save.

Most checkpoint files include a VAE. However, the baked-in VAE of some older SD 1.5 models produces slightly washed-out or oversaturated colors. A common fix is to load a separate, higher-quality VAE file and use it in place of the checkpoint's built-in one.

To use a standalone VAE: add a Load VAE node (right-click → Add Node → loaders → Load VAE), select the VAE file from its dropdown, and connect its output to the VAE input of the VAE Decode node, disconnecting the VAE connection from Load Checkpoint. The most commonly recommended standalone VAE for SD 1.5 is vae-ft-mse-840000-ema-pruned.safetensors, available on Hugging Face.

For SDXL and FLUX.1, the baked-in VAE is generally good quality and does not need to be replaced.

CLIP: How Text Becomes Numbers the Model Can Use

CLIP (Contrastive Language-Image Pre-training) is a text encoder. Its job is to take your prompt text and convert it into a dense numerical vector — a long sequence of numbers that captures the semantic meaning of your words in a way the diffusion UNet understands.

This encoding step happens in the CLIP Text Encode node. The output is labeled CONDITIONING rather than CLIP — the conditioning is the encoded representation of your prompt, ready to guide the KSampler.

SD 1.5 uses a single CLIP encoder (ViT-L/14). SDXL uses two encoders in parallel: clip_g (OpenCLIP ViT-bigG) and clip_l (ViT-L/14). You can see this in the SDXL default workflow, which has two CLIP Text Encode nodes connected to a separate CLIPTextEncodeSDXL node that merges them.

Dual CLIP in FLUX.1

FLUX.1 uses an entirely different CLIP setup: T5-XXL and CLIP-L, loaded separately rather than bundled into the checkpoint. When running FLUX.1 workflows, you will see a DualCLIPLoader node (or two separate CLIP loaders) instead of the standard single Load Checkpoint approach.

T5-XXL is a much larger text encoder than anything used in SD 1.5 or SDXL — it understands complex, natural language descriptions rather than tag-based prompts. This is why FLUX.1 prompts are written differently: full sentences describing the scene in detail, rather than comma-separated style tags. The T5-XXL encoder handles the nuance of natural language much more accurately.

Conditioning: Positive and Negative

The KSampler has two conditioning inputs: positive and negative. Both are CONDITIONING data from CLIP Text Encode nodes.

Positive conditioning tells the diffusion process what concepts to move toward during denoising. The more specific and detailed your positive prompt, the more strongly these concepts appear in the output.

Negative conditioning tells the process what concepts to move away from. For SD 1.5 and SDXL, negative prompts are highly effective — common terms like blurry, low quality, deformed, ugly, watermark reliably suppress those qualities.

For FLUX.1, the T5-XXL encoder produces conditioning vectors that the model uses differently. Negative prompts in FLUX.1 workflows have significantly less effect on the output. Most FLUX.1 users either leave the negative prompt empty or use a minimal one. Focus your creative effort on the positive prompt when working with FLUX.

Understanding these three systems — VAE, CLIP, and conditioning — gives you a complete mental model of what is happening inside every generation. In the next lesson you will see how the KSampler uses all of this to actually run the diffusion process.