IPAdapter: Image-Based Style Transfer

Style Transfer Without Retraining

IPAdapter (Image Prompt Adapter) lets you steer image generation using a reference photo instead of — or alongside — a text prompt. Rather than describing a visual style in words, you show ComfyUI an example and it extracts the visual language from that image and injects it into your generation through the model's attention layers.

The practical effect: load a reference image of a specific painting, and your generated images adopt that painting's color palette, brushstroke quality, and lighting style. Load a portrait photo, and your outputs carry that person's facial features into new scenes. Load a product render, and your generated images match that object's material and lighting.

IPAdapter is different from ControlNet. ControlNet controls composition and structure (where things are arranged). IPAdapter controls appearance and style (what things look like). You can and should combine them: ControlNet sets the pose or layout, IPAdapter sets the visual treatment.

Installing IPAdapter Plus

IPAdapter Plus is not included in standard ComfyUI. Install it through ComfyUI-Manager:

1. Open Manager → Install Custom Nodes
2. Search for "ComfyUI_IPAdapter_plus" and install it
3. Restart ComfyUI

You also need IPAdapter model files. These are separate from your checkpoint and go in ComfyUI/models/ipadapter/. Download them from the IPAdapter Plus GitHub releases page (link available in the repo's README). Download the model that matches your checkpoint architecture — there are separate IPAdapter models for SD 1.5 and SDXL.

The Basic IPAdapter Workflow

The core IPAdapter workflow adds three nodes to your standard text-to-image graph:

1. Load Image: loads your reference image (the style source)
2. IPAdapter Unified Loader: loads the IPAdapter model file and optionally a CLIP vision model. Set the preset to match your architecture (SD 1.5 or SDXL).
3. Apply IPAdapter: takes model, IPAdapter (from the loader), and image (from Load Image) as inputs. Outputs a modified MODEL that carries the reference image's visual influence.

Connect the Apply IPAdapter output (MODEL) to your KSampler's model input, replacing the direct connection from Load Checkpoint. Your text prompt still applies alongside the image reference — the two sources of guidance blend together during sampling.

Weight and Weight Type

The Apply IPAdapter node has two key parameters:

• weight: overall strength of the image reference (0.0–2.0, start at 0.6–0.8)
• weight_type: how the reference image's influence is applied

Weight type options:

• linear (default): balanced style influence across the image
• style transfer: stronger pull toward the reference image's aesthetic; good for matching a painting or illustration style
• composition: copies the compositional structure of the reference more than its texture
• strong style transfer: maximum style influence; use carefully at lower weight values

For most style transfer work, start with linear at weight 0.7. If the style isn't coming through strongly enough, switch to style transfer weight_type and lower the weight to 0.5–0.6 to avoid overwhelming your prompt.

The start_at and end_at parameters (0.0–1.0) control during which portion of the sampling process IPAdapter is active. Setting end_at to 0.5 means IPAdapter influences the early diffusion steps (structure and style) but not the final detail pass — useful for a subtler effect.

FaceID Variant for Portrait Identity

The standard IPAdapter copies visual style. The FaceID variant goes further — it copies a specific person's facial identity from a reference photo into your generated portraits. This is the closest thing in local image generation to "generate an image of this specific person in a new scene."

FaceID requires two additional items beyond standard IPAdapter:

• InsightFace: a face analysis library. Install it via ComfyUI-Manager or via pip (pip install insightface). The AntelopeV2 model files go in ComfyUI/models/insightface/models/antelopev2/.
• IPAdapter FaceID model: a specific version of the IPAdapter model trained on face identity. Download from Hugging Face (h94/IP-Adapter-FaceID).

The FaceID workflow uses two different nodes:

• IPAdapterUnifiedLoaderFaceID: replaces the standard Unified Loader; set the preset to "FACEID" or "FACEID PLUS" depending on which model you downloaded
• IPAdapterFaceID: replaces Apply IPAdapter; takes the face reference image and applies identity conditioning

Load a clean, well-lit, forward-facing reference photo for best results. Weight 0.7–0.9 gives strong identity matching. Use FaceID alongside a descriptive text prompt to place the person in a new scene or style while maintaining their appearance.