Learn › ComfyUI Advanced: Training, Kontext, and Automation › FLUX Fill and FLUX Depth: Advanced Inpainting Control

FLUX Fill and FLUX Depth: Advanced Inpainting Control

Advanced 🕐 13 min Lesson 3 of 12
What you'll learn
  • Set up the FLUX.1 Fill Dev model and use the official inpainting template to fill a masked region
  • Draw accurate masks using ComfyUI's MaskEditor for inpainting and outpainting workflows
  • Install BiRefNet via ComfyUI-Manager and use it to automatically segment a subject for background replacement
  • Explain the difference between FLUX Depth and FLUX Canny and choose the correct tool for a given editing task

Beyond Standard Inpainting

The inpainting workflow covered in ComfyUI Image Generation uses your base model — FLUX, SDXL, or SD 1.5 — to re-generate a masked region. The result depends heavily on your prompt, the denoise strength, and how well the base model can infer what belongs in that region from context. For simple fills it works, but for complex scenes where the fill needs to blend seamlessly with lighting, perspective, and surrounding content, the base model often requires significant iteration.

FLUX Fill is a dedicated inpainting model, not the base FLUX model with a mask applied. It was specifically trained on the task of context-aware region filling — understanding the content surrounding the mask and generating output that integrates with it. The result is cleaner fills with less prompt-engineering overhead, particularly for backgrounds, environmental extensions, and subject removal.

FLUX Depth and FLUX Canny are different tools for a different problem: structure-preserving generation. Rather than filling a region based on context, they take a depth map or edge map from an existing image and use it as hard conditioning — generating new content that follows the same 3D structure or the same edge outlines as the original, while changing the appearance according to your prompt. These are the tools to reach for when you need to restyle an image but cannot allow the composition or spatial structure to change.

FLUX Fill: Setup

FLUX Fill requires one additional model file. If you already have FLUX text-to-image working, you have all the supporting files — only the diffusion model is new:

  • flux1-fill-dev.safetensors → ComfyUI/models/diffusion_models/
    Source: Hugging Face — black-forest-labs/FLUX.1-Fill-dev (requires accepting the usage agreement before downloading)

The text encoders (clip_l.safetensors, t5xxl_fp16.safetensors) and VAE (ae.safetensors) are the same files used by FLUX.1 [dev] — they live in text_encoders/ and vae/ as before. No new supporting files are needed.

Once the model is in place, load the official template from Workflow → Browse Templates → Flux and select the inpainting or outpainting example. The templates use the same core node structure as FLUX text-to-image (Load Diffusion Model, DualCLIPLoader, Load VAE, Load Image) with the mask handling added.

Drawing Masks with MaskEditor

The region you want FLUX Fill to regenerate is defined by a mask — a black and white image the same size as your input, where white marks what to fill and black marks what to preserve. ComfyUI's built-in MaskEditor lets you draw masks directly on the loaded image without leaving the application.

To use MaskEditor: right-click the Load Image node and select Open in MaskEditor. A brush tool appears. Paint white over the region you want to replace. Click Save to store the mask and return to the main canvas. The mask output from Load Image will now carry your drawn selection into the Fill workflow.

Tips for clean masks:

  • Extend slightly beyond the edge of what you want to remove. A mask that sits exactly on an object's edge tends to leave a visible seam; extending a few pixels into the surrounding content gives Fill more context to blend from.
  • Use a soft brush for organic subjects (hair, fabric edges, foliage) and a hard brush for clean geometric objects. The mask edge influences how the fill blends with the unmasked region.
  • Keep the unmasked region informative. FLUX Fill reads the surrounding pixels to infer what belongs in the masked area. If your mask covers most of the image, the fill becomes more of a generation than a contextual fill and quality drops. Masks under 40% of the image area generally produce the best results.

Inpainting: Removing and Replacing Content

The core FLUX Fill inpainting workflow takes three inputs: the source image, the mask you drew, and a text prompt describing what should appear in the masked region. If you want to remove an object entirely and have the background fill in naturally, leave the prompt empty or write a brief description of the surrounding environment (e.g., "brick wall continuation"). If you want to replace the masked content with something specific, describe it in the prompt.

FLUX Fill reads the surrounding pixels to infer lighting direction, texture, perspective, and colour balance. A prompt of "brick wall" in a masked region surrounded by brick will produce a wall that matches the existing masonry, not a generic brick texture. This context-awareness is what distinguishes it from standard inpainting.

For subject removal, the typical workflow is: mask the subject tightly, leave the prompt empty, run Fill. The model infers the background from the surrounding content. For complex backgrounds with strong patterns (tiles, cobblestones, patterned fabric) it may require a prompt hint to match the pattern type, but for natural backgrounds like grass, sky, and walls it generally fills accurately without one.

Outpainting: Extending Beyond the Frame

FLUX Fill outpainting extends an image beyond its original borders. The official outpainting template handles the padding and mask creation automatically — you specify how much to extend in each direction and the template generates the appropriate mask and canvas.

Outpainting reads the original image edges to infer what the extended scene should look like. For best results, write a prompt that describes the full extended scene, not just the new area: "a sunlit forest path extending to the left" rather than just "trees." Including context from the existing image in the prompt helps Fill maintain consistency between the original and the extension.

Outpainting in multiple small steps — extending by 20–30% of the image width at a time rather than trying to double the canvas in one pass — produces more coherent results, particularly when extending a complex scene with strong perspective lines.

Automatic Segmentation with BiRefNet

Drawing masks by hand is accurate but slow for subjects with complex edges — hair, fur, foliage, or fine clothing details. BiRefNet is a segmentation model that automatically extracts a precise subject mask from an image, eliminating most manual masking work for well-defined subjects.

BiRefNet is available as a ComfyUI custom node. Install it via ComfyUI-Manager by searching for BiRefNet. The workflow is straightforward:

  1. Load your source image in a Load Image node.
  2. Pass the image to the BiRefNet node. It outputs a binary mask isolating the main subject.
  3. Connect the mask output to the FLUX Fill workflow's mask input, replacing the MaskEditor output.
  4. Optionally invert the mask using an InvertMask node if you want to edit the subject rather than the background (BiRefNet masks the foreground by default).

BiRefNet works well on subjects with clear contrast from their background — portraits, isolated objects, animals. It is less reliable on subjects that blend into the background, transparent objects, or heavily cluttered scenes. For those cases, manual masking or a combination of BiRefNet plus manual cleanup in MaskEditor produces better results.

The BiRefNet + FLUX Fill combination is particularly effective for background replacement: BiRefNet isolates the subject automatically, the inverted mask selects the background, and FLUX Fill generates a new background that integrates with the subject's lighting.

FLUX Depth: Structure-Preserving Generation

FLUX Depth takes a depth map — a greyscale image where brighter pixels represent objects closer to the camera and darker pixels represent objects further away — and uses it as hard conditioning. The generated image follows the same 3D spatial arrangement as the original, regardless of what the text prompt describes.

This makes FLUX Depth the right tool when you need to restyle an image completely (change the art style, the lighting, the season, the materials) while keeping the composition and spatial arrangement exactly as they are. Restyling a portrait from photorealistic to oil painting, converting a building photograph to a watercolour architectural sketch, applying a different time of day to a landscape — all while ensuring nothing moves in space.

Setup: Download the FLUX.1 Depth [dev] model from Black Forest Labs on Hugging Face and place it in diffusion_models/ alongside your other FLUX models. The workflow uses the same text encoders and VAE as FLUX.1 [dev].

Depth map generation: You need a depth map to match your source image. Load the source image into a MiDaS Depth Estimator preprocessor node (available via ComfyUI-Manager, in the ControlNet Auxiliary Preprocessors pack). The node outputs a greyscale depth map. Connect that depth map, your text prompt, and your source image into the FLUX Depth workflow.

Key setting: The strength parameter on the depth conditioning node controls how strictly the output follows the depth map. At 1.0 the spatial structure is preserved exactly; at lower values the composition may drift toward the text prompt. For pure restyling, keep strength at 0.9 or above.

FLUX Canny: Edge-Map Guided Generation

FLUX Canny works similarly to Depth, but conditions on a Canny edge map — a line drawing that captures the outlines and fine structural details of the original image — rather than a depth map. Where Depth preserves spatial arrangement, Canny preserves the exact shapes, fine details, and line structure of the original.

Canny conditioning is better than Depth for images where fine edge details matter: architectural drawings, product shots with intricate geometry, illustrated characters, or any image where the specific outlines are more important than the 3D spatial arrangement.

Depth vs Canny — when to choose each:

  • Use Depth when you want to preserve the 3D composition (foreground/background relationship, spatial depth, perspective) while changing everything about the appearance.
  • Use Canny when you want to preserve specific lines, outlines, and fine structural details while changing the aesthetic.
  • For portrait restyling: Depth is usually the better choice — it maintains the face's spatial relationship without locking every edge.
  • For product or architectural restyling: Canny is usually better — precise outlines matter more than spatial depth.

Both models use the same workflow structure and the same LoadFluxControlNet + ApplyFluxControlNet nodes, with the only difference being which preprocessing node generates the conditioning map. This makes it easy to switch between them by swapping the preprocessor and the model file, leaving everything else in the workflow unchanged.

Key takeaways
  • FLUX Fill is a dedicated inpainting model, not the base model with a mask applied — it reads surrounding pixels to infer context, producing fills that integrate with the existing scene rather than generating from prompt alone
  • Only flux1-fill-dev.safetensors needs to be downloaded if you already have FLUX text-to-image working — the same text encoders and VAE are reused
  • BiRefNet automatically segments the main subject and outputs a binary mask, eliminating hand-drawing for portraits and isolated objects — install via ComfyUI-Manager
  • FLUX Depth preserves 3D spatial arrangement; FLUX Canny preserves fine edges and outlines — for portrait restyling use Depth, for product and architectural restyling use Canny
  • Masks covering under 40% of the image produce the most coherent FLUX Fill results — larger masks turn inpainting into generation and reduce context-awareness
Learn more every week Get 5 hand-picked prompts + new AI tools every Monday. Free.