LoRA Training, Part 3: Testing and Fixing Your LoRA
- Load a trained LoRA in ComfyUI and run the four-category test protocol to evaluate generalisation
- Identify all five common LoRA failure modes from their symptoms
- Apply the correct fix for each failure mode: adjust strength, adjust prompt, or retrain with specific changes
- Combine a character LoRA with FLUX Kontext for stronger character consistency in edited scenes
Loading Your LoRA in ComfyUI
Copy the trained .safetensors file into ComfyUI/models/loras/. If ComfyUI is already running, right-click any Load LoRA node and select Refresh to see the new file. Add a Load LoRA node between your Load Diffusion Model output and the KSampler model input, with the DualCLIPLoader output also passing through the same Load LoRA node. Connect both model and clip outputs to the KSampler.
Start testing at LoRA strength 0.8. This is not a universal recommendation — it is a starting point for evaluation. You will adjust it after seeing how the LoRA activates.
The Test Protocol
Testing a LoRA against only the prompt types used during training tells you very little. You need to test against prompts the LoRA has not seen, in settings that differ from your training images. This is what distinguishes good generalisation from memorisation.
Run these four test categories with your trigger word in each prompt:
- Novel backgrounds: Place the character in settings not present in any training image. If you trained on indoor images, test outdoors. If you trained on plain backgrounds, test in complex environments. A well-trained LoRA maintains character identity in unfamiliar contexts.
- Novel poses: Request poses not represented in your dataset. If all training images were standing poses, test sitting, crouching, or action poses. A well-trained LoRA produces the character reliably in new poses; an over-trained one will try to replicate a training pose regardless of your prompt.
- Style changes: Apply art style modifiers — "oil painting," "anime style," "charcoal sketch" — alongside the trigger word. A good character LoRA maintains recognisable identity while adapting to the requested style.
- Trigger word removed: Generate the same prompt without the trigger word. The output should look like a generic person in that style, not your character. If your character appears strongly without the trigger word, the LoRA has contaminated the checkpoint — this is an overfitting symptom.
The Five Common Failure Modes
1. Under-activation — the character is barely recognisable
Symptom: the trigger word produces a vague resemblance at best; the character barely differs from a prompt without the trigger word.
Causes: too few training steps, learning rate too low, trigger word too similar to common vocabulary.
Quick fix (no retraining): increase LoRA strength from 0.8 toward 1.0–1.2 and test again.
Proper fix: retrain with more steps (increase repeats), a higher learning rate (try 2e-4), or a more distinctive trigger word.
2. Over-training — the trigger word overrides everything
Symptom: the trigger word dominates the generation regardless of other prompt elements; "jksmith2024, standing in a forest" produces the character lying down because that was a common training pose; changing style keywords has little effect.
Causes: too many training steps, learning rate too high, too few unique images in the dataset.
Quick fix: reduce LoRA strength to 0.5–0.6 and test again. This often recovers reasonable flexibility without retraining.
Proper fix: retrain with fewer steps (stop at the 500 or 1000-step checkpoint rather than the final), a lower learning rate, or a larger dataset with more varied images.
3. Attribute bleed — wrong attributes appearing outside the character
Symptom: a colour or texture associated with the character (the red jacket, the blue eyes) bleeds into the background or other elements in the scene.
Causes: strong unique attributes in the training images that were not properly described in captions, so the model associated them with the trigger word globally rather than with specific parts of the subject.
Quick fix: add negative prompt terms that describe the bleeding attribute (e.g., "red jacket" in the negative if red is spreading into the background).
Proper fix: retrain with better captions that explicitly describe the clothing and accessories in every caption, so the model localises them to the subject rather than the trigger word globally.
4. Trigger word not activating at low strength
Symptom: the character only appears at high LoRA strength (0.9+), which causes other problems like artifact generation or stiffness.
Causes: rank too low, trigger word appears infrequently in captions (check that every caption starts with it), or competing strong associations in the base model.
Quick fix: use the trigger word earlier in the prompt and try using it twice ("jksmith2024 portrait of jksmith2024").
Proper fix: retrain with rank 32 instead of 16, and confirm every caption file starts with the trigger word.
5. Identity drift in novel poses
Symptom: the character looks correct in head-on views but drifts to a different appearance in profile or three-quarter views.
Causes: training dataset lacked angle diversity — most or all images showed the character from a single angle.
No quick fix: this requires dataset improvement. Add more images with varied angles and retrain.
Comparing Checkpoint Files
If you saved checkpoints every 250–500 steps during training (as recommended in Lesson 7), you can compare them to find the best stopping point. Load each checkpoint LoRA and run the same test prompt against all of them. The pattern you are looking for:
- Early checkpoints (500 steps): character is recognisable but weak, details not crisp
- Middle checkpoints (1,000–1,500 steps): character is accurate and flexible, responds well to varied prompts
- Late checkpoints (2,000+ steps): character is accurate but may start losing flexibility — the trigger word starts to override other prompt elements
The sweet spot is usually the checkpoint just before overfitting begins. If your 1,000-step checkpoint produces better results than the final model, that is your LoRA — use it.
Adjusting Strength in Production
LoRA strength is not a one-time setting. The right value depends on what else is in the workflow:
- LoRA alone: 0.7–0.9 for most character LoRAs
- LoRA + ControlNet: reduce to 0.6–0.75 — ControlNet and LoRA compete for influence; both at full strength produce artifacts
- LoRA + IPAdapter: reduce to 0.5–0.65 — IPAdapter also drives character appearance, so the combined influence needs to be balanced
- LoRA + Kontext editing: reduce to 0.5–0.7 for editing passes — Kontext needs freedom to follow the edit instruction, but the LoRA keeps the character anchored
Combining Your LoRA With Kontext
A trained character LoRA and FLUX Kontext are complementary tools. The LoRA provides a strong trained prior for the character's appearance; Kontext provides instruction-following editing. Together, they give you the strongest character consistency available without the Pro/Max API:
- Generate a base image of the character using the LoRA at strength 0.8. This produces a clean, accurate starting image.
- Load that image into the Kontext Dev workflow (from Lesson 1). Also load the LoRA at reduced strength (0.6).
- Write a Kontext instruction to change the scene or costume. The LoRA anchors the character's identity while Kontext applies the change.
- Use the output as the reference for the next scene variation, keeping the LoRA loaded throughout the session.
This workflow produces significantly more consistent results than either tool alone — the LoRA prevents identity drift across editing passes, and Kontext prevents the generation from converging back toward training images in new scenes.
When to Retrain vs Adjust
Not every LoRA problem requires retraining. Here is a clear decision guide:
- Adjust strength: over-activation or under-activation where the character is recognisable. Retrain only if the range of usable strength is too narrow (the LoRA is good at 0.9 but causes artifacts at 0.95, leaving no usable range).
- Retrain with more steps: under-training where the trigger word is barely recognisable at any strength.
- Retrain with better dataset: identity drift in novel poses (need angle variety), attribute bleed (need better captions), or the character looks like a blend of training images rather than a consistent person (need more diverse images).
- Retrain with lower rank or steps: over-training where the LoRA overrides other prompt elements even at 0.5 strength.
Retraining on Vast.ai costs under $1.50 and takes 1–2 hours. It is cheap enough that iterating on the dataset and parameters is always preferable to trying to work around a structurally flawed LoRA with prompt engineering.
- Test a LoRA against novel backgrounds, novel poses, style changes, and without the trigger word — generalization to unseen prompts is the real quality test, not reproduction of training images
- Over-training can often be fixed without retraining by reducing LoRA strength to 0.5-0.6; under-training requires either more steps or a higher learning rate
- Checkpoint comparison reveals the ideal stopping point — the best checkpoint is usually where the character is accurate but still flexible, not the final file from a full training run
- Retraining on Vast.ai costs under $1.50 and takes 1-2 hours — iterating on dataset quality is always preferable to working around a structurally flawed LoRA with prompt tricks
- LoRA strength should be reduced when combining with ControlNet (0.6-0.75), IPAdapter (0.5-0.65), or Kontext editing (0.5-0.7) to prevent competing influences from causing artifacts
Progress is saved in this browser only — clear your cache or switch device and it's gone. Create a free account and it follows you everywhere.