LoRA Training, Part 2: Training Inside Kohya
- Launch the Kohya GUI and configure the three required paths: pretrained model, image folder, and output folder
- Set the correct FLUX LoRA training parameters: rank 16, alpha 16, learning rate 1e-4, AdamW8bit optimizer, cosine scheduler, bf16 precision
- Read the training loss log and identify healthy training progress versus warning signs
- Download the finished LoRA file and place it in ComfyUI for use
Kohya vs ComfyUI-FluxTrainer
Two training tools are commonly used for FLUX LoRA training, and both run inside a browser interface:
- Kohya_ss: The original and most widely documented training toolkit for Stable Diffusion models. Mature, well-tested, with a large community and extensive documentation. The recommended choice for first-time LoRA training because most guides and troubleshooting resources assume it.
- ComfyUI-FluxTrainer: A custom node that wraps Kohya's FLUX training scripts inside a ComfyUI workflow. Install via ComfyUI-Manager. Good if you are already comfortable with ComfyUI workflows and prefer to stay in that environment. Under the hood, it runs the same training scripts as Kohya — the interface is different, the results are equivalent.
This lesson covers Kohya. If you are on Vast.ai, look for a template that includes Kohya pre-installed — it saves 1–2 hours of environment setup. If you need to install Kohya manually, follow the official README at the kohya_ss GitHub repository.
Launching the Kohya Interface
Once Kohya is installed on your instance, launch it by running python kohya_gui.py from the Kohya directory (or the launch script your template provides). The GUI runs on a local port, typically 7860. If you are on a remote GPU, you will need to forward that port over SSH:
ssh -L 7860:localhost:7860 -p PORT root@HOSTNAME
Open http://localhost:7860 in your local browser. The Kohya GUI loads. Navigate to the LoRA tab at the top, then select Training.
Pointing Kohya at Your Model and Dataset
Three paths must be set before anything else:
- Pretrained model name or path: The full path to your FLUX.1 [dev] model file. If you are on Vast.ai with a training template, this may already be pre-filled. If not, download
flux1-dev.safetensorsfrom Black Forest Labs on Hugging Face and point this field to its location on the instance. - Image folder: The path to the
img/folder you prepared in Lesson 6 — the one containing your20_jksmith2024/subfolder (or whatever your repeat-count and trigger-word folder is named). - Output folder: Where the trained LoRA files will be saved. Set this to a dedicated folder like
/workspace/output/so you can find the files easily when training completes.
Key Training Parameters for FLUX
The following settings have been tested for FLUX.1 LoRA training. The defaults in Kohya were designed for earlier Stable Diffusion models — do not use them unchanged for FLUX.
Network Rank (dim): Set to 16. For FLUX, 16–32 is the recommended range. Rank controls the size and expressiveness of the LoRA — higher rank can store more information but raises the risk of overfitting on small datasets. Start at 16; increase to 32 only if your first run looks under-trained after testing.
Network Alpha: Set to 16 (same as rank). Alpha is a scaling factor that determines the effective learning strength. Setting alpha equal to rank is the standard starting configuration — it means the LoRA weights are applied at full strength. Lower alpha values reduce the effective training intensity, which can help prevent overfitting but may require more steps to achieve the same result.
Learning Rate: Set to 1e-4 (which is 0.0001). FLUX is more sensitive to learning rate than SDXL. Too high (above 2e-4) produces an over-trained, unstable LoRA; too low (below 5e-5) requires many more steps to converge. Start at 1e-4.
Text Encoder Learning Rate: Set to the same as the learning rate (1e-4) or slightly lower (5e-5). The text encoder learns to associate your trigger word with the training images, so it needs to train, but at a conservative rate.
Optimizer: Select AdamW8bit. This is the VRAM-efficient version of AdamW and is the standard choice for training on a 24 GB GPU. If you have 12 GB VRAM and are using block swapping or NF4 quantization, Adafactor (which adjusts learning rate automatically) can be a better fit.
LR Scheduler: Select cosine. Cosine scheduling gradually reduces the learning rate from its starting value toward 0 over the training session. This produces smoother convergence than a constant rate. Set LR warmup ratio to 0.1 (10% of training steps are used to ramp up from 0 to the full learning rate).
Mixed Precision: Set to bf16. BFloat16 is the correct precision mode for FLUX.1 training — do not use fp16 (which is correct for SDXL but can produce NaN errors with FLUX).
Max Resolution: Set to 1024,1024. This tells Kohya the maximum dimensions for training images — images larger than this will be downscaled. All your dataset images should already be 1024px.
Batch Size: Set to 1. Increasing batch size speeds up training but requires proportionally more VRAM. At batch size 1 on a 24 GB GPU with FLUX, you are already close to the VRAM limit. Leave this at 1 unless you have 48 GB available.
Gradient Checkpointing: Enable this. Gradient checkpointing reduces VRAM usage during training by recalculating intermediate activations on the backward pass rather than storing them. Training is slightly slower but VRAM usage drops significantly.
Save every N steps: Set to 250 or 500. This saves a checkpoint LoRA file every 250 or 500 steps, in addition to the final file. These intermediate checkpoints are valuable for diagnosing overfitting — you can test the 500-step and 1500-step checkpoints against each other to see where the model peaked before overfitting began.
Epochs: Calculate from your step target. For a 20-image dataset with 20 repeats targeting 2,000 steps: 2,000 ÷ (20 images × 20 repeats) = 5 epochs. Enter that value. The step count in the training log should reach approximately your target when training finishes.
Starting the Training Run
Once all parameters are set, click Train model. The training log appears below the button. What you should see in the first few minutes:
- Model loading messages — FLUX is a large model and takes 1–2 minutes to load on most GPUs
- Dataset loading and preprocessing — you will see your image filenames appear
- The first step loss value — typically in the range of 0.1 to 0.5 at the start
If training stops immediately with an out-of-memory error, enable gradient checkpointing if you have not already, reduce batch size to 1, and check that no other GPU processes are running on the instance.
Reading the Loss Log
The training log shows a loss value at each step. Loss measures how wrong the model's output is compared to the training images — lower loss means the model is better at reproducing the training data.
What healthy loss progression looks like:
- Start: Loss is relatively high (0.1–0.4), decreasing steadily each step
- Middle: Loss continues to decrease but more slowly, eventually plateauing
- Plateau: Loss stops meaningfully decreasing — this is where training is effectively complete
Warning signs:
- Loss spikes wildly: Learning rate is too high. Restart with a lower value (try 5e-5).
- Loss never decreases at all: Dataset path is wrong, or the model file is not loading correctly. Check the error messages above the loss values.
- Loss keeps decreasing very slowly past 3,000 steps without plateauing: Learning rate may be too low. The LoRA is probably fine — test it at 2,000 steps.
You do not need to monitor the entire run. Start training, check that the first 50 steps look reasonable (decreasing loss, no crashes), then let it run. A 2,000-step run takes approximately 1–2 hours on a 24 GB GPU at batch size 1 with FLUX.
Downloading the Finished LoRA
When training completes, Kohya saves a file named last.safetensors (and any checkpoint files) in your output folder. The finished LoRA is typically 70–300 MB depending on rank and training duration.
Download it to your local machine using scp:
scp -P PORT root@HOSTNAME:/workspace/output/last.safetensors ./my-lora.safetensors
Once downloaded, copy the file into ComfyUI/models/loras/ on your local machine. Right-click any Load LoRA node in ComfyUI and select Refresh to see the new file in the dropdown.
After downloading, stop or destroy the Vast.ai instance to stop storage charges. Keep the finished .safetensors file locally — training again from scratch is cheap (under $1.50), but losing the LoRA file means starting over.
Lesson 8 covers testing your trained LoRA: how to evaluate whether it activates correctly, what the common failure modes look like, and how to fix them without retraining from scratch.
- Set mixed precision to bf16 for FLUX training, not fp16 — fp16 is correct for SDXL but can produce NaN errors with FLUX
- Network rank 16 and alpha 16 are the correct FLUX starting settings; alpha equal to rank applies weights at full strength
- Save checkpoints every 250-500 steps — these intermediate files let you compare training stages and find the best stopping point without retraining
- Learning rate 1e-4 is the FLUX starting point; too high (above 2e-4) produces instability, too low (below 5e-5) requires excessive steps
- A healthy loss curve decreases steadily then plateaus — wild spikes mean the learning rate is too high; a plateau reached at 1500-2000 steps means training is complete
Progress is saved in this browser only — clear your cache or switch device and it's gone. Create a free account and it follows you everywhere.